### Install Floki in Livebook or Script
Source: https://github.com/philss/floki/blob/main/README.md
Provides instructions for installing Floki using `Mix.install/2`, suitable for use in Livebook or standalone scripts. This method avoids modifying the `mix.exs` file.
```elixir
Mix.install([
{:floki, "~> 0.38.0"}
])
```
--------------------------------
### Complete Floki Configuration Example
Source: https://github.com/philss/floki/blob/main/_autodocs/configuration.md
A comprehensive example of Floki configuration, setting the HTML parser, raw HTML encoding, and self-closing tags. Includes test-specific configuration for the parser.
```elixir
# config/config.exs
config :floki,
html_parser: Floki.HTMLParser.Html5ever,
encode_raw_html: true,
self_closing_tags: [
"area", "base", "br", "col", "embed", "hr", "img", "input",
"keygen", "link", "meta", "param", "source", "track", "wbr"
]
# config/test.exs (test-specific)
import Config
config :floki,
html_parser: Floki.HTMLParser.Mochiweb
```
--------------------------------
### Example HTML Comments
Source: https://github.com/philss/floki/blob/main/_autodocs/types.md
Shows examples of HTML comment nodes.
```elixir
{:comment, " This is a comment "}
{:comment, " TODO: implement pagination "}
```
--------------------------------
### Floki Global Configuration
Source: https://github.com/philss/floki/blob/main/_autodocs/REFERENCE.md
Examples of global configuration settings for Floki in config/config.exs.
```elixir
# Choose HTML parser
config :floki, :html_parser, Floki.HTMLParser.FastHtml
# Control encoding in raw_html
config :floki, :encode_raw_html, true
# Customize self-closing tags
config :floki, :self_closing_tags, ["br", "hr", ...]
```
--------------------------------
### Example HTML Doctype Declarations
Source: https://github.com/philss/floki/blob/main/_autodocs/types.md
Provides examples of DOCTYPE declarations, including HTML5 and XHTML formats.
```elixir
{:doctype, "html", "", ""} # HTML5
{:doctype, "html", "-//W3C//DTD XHTML 1.0 Transitional//EN", "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"}
```
--------------------------------
### Example HTML Tag Structures
Source: https://github.com/philss/floki/blob/main/_autodocs/types.md
Shows various examples of HTML tag structures, including elements with attributes, nested elements, and empty elements.
```elixir
{"div", [{"id", "content"}], ["Hello"]}
{"p", [{"class", "intro"}], ["Welcome"]}
{"a", [{"href", "http://example.com"}], ["Link"]}
{"div", [], []} # Empty div
{"ul", [], [{"li", [], ["Item 1"]}, {"li", [], ["Item 2"]}]}
```
--------------------------------
### Example HTML Declarations
Source: https://github.com/philss/floki/blob/main/_autodocs/types.md
Illustrates the structure of XML processing instructions, including the target and attributes.
```elixir
{:pi, "xml", [{"version", "1.0"}, {"encoding", "utf-8"}]}
{:pi, "xml-stylesheet", [{"href", "style.xsl"}]}
```
--------------------------------
### Example HTML Attributes Map
Source: https://github.com/philss/floki/blob/main/_autodocs/types.md
Provides an example of an HTML attributes map.
```elixir
%{"class" => "alert warning", "data-level" => "high"}
```
--------------------------------
### Example HTML Attributes
Source: https://github.com/philss/floki/blob/main/_autodocs/types.md
Illustrates the string tuple format for HTML attributes.
```elixir
{"class", "container"}
{"id", "main"}
{"data-count", "42"}
```
--------------------------------
### Example HTML Text Nodes
Source: https://github.com/philss/floki/blob/main/_autodocs/types.md
Illustrates plain text content nodes, including whitespace.
```elixir
"Hello, World!"
"Some text content"
" " # Whitespace text node
```
--------------------------------
### HTML Tree Examples
Source: https://github.com/philss/floki/blob/main/_autodocs/types.md
Illustrates different forms of an HTML tree, including a parsed document, a parsed fragment, and an empty tree.
```elixir
# Parsed document
[{"html", [], [{"head", [], []}, {"body", [], ["Content"]}]}]
```
```elixir
# Parsed fragment
[{"p", [], ["Hello"]}, {"p", [], ["World"]}]
```
```elixir
# Empty tree
[]
```
--------------------------------
### Floki Combinators
Source: https://github.com/philss/floki/blob/main/_autodocs/REFERENCE.md
Examples of using CSS combinators with Floki's find function.
```elixir
find(tree, "div p") # Descendant
find(tree, "div > p") # Child
find(tree, "p + p") # Adjacent sibling
find(tree, "p ~ p") # General sibling
```
--------------------------------
### Parse Document with Mochiweb
Source: https://github.com/philss/floki/blob/main/_autodocs/api-reference/html_parser.md
Example of parsing an HTML document using the Mochiweb parser explicitly.
```elixir
Floki.parse_document(html, html_parser: Floki.HTMLParser.Mochiweb)
```
--------------------------------
### Floki Pseudo-Classes
Source: https://github.com/philss/floki/blob/main/_autodocs/REFERENCE.md
Examples of using pseudo-classes with Floki's find function for selecting specific elements.
```elixir
find(tree, ":first-child") # First child
find(tree, ":last-child") # Last child
find(tree, ":nth-child(2)") # nth child
find(tree, ":nth-last-child(2)") # nth from last
find(tree, ":first-of-type") # First of type
find(tree, ":last-of-type") # Last of type
find(tree, ":nth-of-type(2)") # nth of type
find(tree, ":checked") # Checked input
find(tree, ":disabled") # Disabled element
find(tree, ":not(p)") # Negation
find(tree, ":has(a)") # Has child
find(tree, ":root") # Root element
find(tree, ":fl-contains('text'"
find(tree, ":fl-icontains('foo'"
```
--------------------------------
### Parse Document with FastHtml
Source: https://github.com/philss/floki/blob/main/_autodocs/api-reference/html_parser.md
Example of parsing an HTML document using the FastHtml parser explicitly.
```elixir
Floki.parse_document(html, html_parser: Floki.HTMLParser.FastHtml)
```
--------------------------------
### CSS Selector Examples
Source: https://github.com/philss/floki/blob/main/_autodocs/types.md
Demonstrates the three ways to specify a CSS selector: as a string, a single parsed struct, or a list of structs for multiple selectors.
```elixir
# String selector
"div.container > p"
```
```elixir
# Parsed selector struct
%Floki.Selector{type: "div", classes: ["container"]}
```
```elixir
# List of selectors (comma-separated)
[
%Floki.Selector{type: "div"},
%Floki.Selector{type: "p"}
]
```
--------------------------------
### Example: Parsing multiple selectors
Source: https://github.com/philss/floki/blob/main/_autodocs/api-reference/selector.md
Shows how to parse a string containing multiple comma-separated selectors, resulting in a list of individual Floki.Selector structs.
```elixir
selectors = Floki.Selector.Parser.parse(".warning, .error")
# => [%Selector{classes: ["warning"]}, %Selector{classes: ["error"]}]
```
--------------------------------
### Selector Optimization Examples
Source: https://github.com/philss/floki/blob/main/_autodocs/api-reference/css_selectors.md
Demonstrates optimized and less optimized selector patterns. More specific selectors and use of child combinators generally lead to better performance.
```elixir
# Fast - specific selector
Floki.find(tree, "article.post > p.intro")
```
```elixir
# Slower - too broad
Floki.find(tree, "p")
```
```elixir
# Fast - use child combinator
Floki.find(tree, "nav > ul > li > a")
```
```elixir
# Slower - unnecessary descendants
Floki.find(tree, "nav ul li a")
```
--------------------------------
### Floki Multiple Selectors
Source: https://github.com/philss/floki/blob/main/_autodocs/REFERENCE.md
Example of using multiple alternative selectors with Floki's find function.
```elixir
find(tree, "h1, h2, h3") # Multiple alternatives
```
--------------------------------
### Immutability Example
Source: https://github.com/philss/floki/blob/main/_autodocs/ARCHITECTURE.md
All Floki operations adhere to immutability, returning new tree structures without modifying the original.
```elixir
# All return new tree, original unchanged
Floki.find(tree, selector)
Floki.traverse_and_update(tree, fun)
Floki.filter_out(tree, selector)
```
--------------------------------
### Example: Parsing a complex selector
Source: https://github.com/philss/floki/blob/main/_autodocs/api-reference/selector.md
Demonstrates parsing a selector string with element type, class, and a child combinator into a structured Floki.Selector representation.
```elixir
selectors = Floki.Selector.Parser.parse("div.container > p.text")
# => [%Floki.Selector{type: "div", classes: ["container"],
# combinator: %Selector.Combinator{match_type: :child,
# selector: %Selector{type: "p", classes: ["text"]}}}]
```
--------------------------------
### Example: Combinator for 'div > p'
Source: https://github.com/philss/floki/blob/main/_autodocs/api-reference/selector.md
Illustrates the resulting Floki.Selector struct when parsing the 'div > p' selector, showing the child combinator and the nested selector.
```elixir
# Selector for "div > p" creates:
%Selector{
type: "div",
combinator: %Selector.Combinator{
match_type: :child,
selector: %Selector{type: "p"}
}
}
```
--------------------------------
### Accumulator Pattern Example
Source: https://github.com/philss/floki/blob/main/_autodocs/ARCHITECTURE.md
Tree traversal functions like `traverse_and_update` support accumulators for stateful operations during the traversal.
```elixir
traverse_and_update(tree, accumulator, fn node, acc -> {new_node, new_acc} end)
```
--------------------------------
### Extract All Text from a Page
Source: https://github.com/philss/floki/blob/main/_autodocs/api-reference/text_extraction.md
Demonstrates a practical example of extracting all text content from a parsed HTML document using `Floki.text/2`.
```elixir
{:ok, page} = Floki.parse_document(html_string)
full_text = Floki.text(page)
# => "all text from all elements"
```
--------------------------------
### Self-Closing Tag Rendering Example
Source: https://github.com/philss/floki/blob/main/_autodocs/configuration.md
Illustrates the effect of self-closing tag configuration on the output of `raw_html/2`. Shows how configured self-closing tags are rendered.
```elixir
Floki.raw_html({"br", [], []})
# => "
" (when "br" is self-closing)
Floki.raw_html({"div", [], []})
# => "
" (not self-closing)
```
--------------------------------
### Delegation Pattern Example
Source: https://github.com/philss/floki/blob/main/_autodocs/ARCHITECTURE.md
The main Floki module delegates common operations to specialized modules like HTMLParser, Finder, and RawHTML.
```elixir
defdelegate parse_document(document, opts \ []), to: Floki.HTMLParser
defdelegate find(tree, selector), to: Floki.Finder
defdelegate raw_html(tree, opts \ []), to: Floki.RawHTML
```
--------------------------------
### Mochiweb Parser Example (Lenient)
Source: https://github.com/philss/floki/blob/main/_autodocs/errors.md
Demonstrates that the default Mochiweb parser often succeeds even with unclosed HTML tags, showing its lenient nature.
```elixir
Floki.parse_document!("")
# Succeeds - unclosed body tag is accepted
```
--------------------------------
### Selector for First Child Element
Source: https://github.com/philss/floki/blob/main/_autodocs/api-reference/selector.md
Example of how a CSS selector like 'p:first-child' is represented internally.
```elixir
# Selector for "p:first-child" creates:
%Selector{
type: "p",
pseudo_classes: [%Selector.PseudoClass{name: "first-child", value: nil}]
}
```
--------------------------------
### Deep Text Extraction Example
Source: https://github.com/philss/floki/blob/main/_autodocs/api-reference/text_extraction.md
Demonstrates how to extract all text deeply from an HTML tree using Floki.text/2, which defaults to DeepText.
```elixir
html = [{"div", [], [{"p", [], ["Hello"]}, " ", {"strong", [], ["World"]}]}]
Floki.text(html) # Uses DeepText internally
# => "Hello World"
```
```elixir
# With separator
Floki.text(html, sep: " | ")
# => "Hello | World"
```
--------------------------------
### Behavior Pattern Example
Source: https://github.com/philss/floki/blob/main/_autodocs/ARCHITECTURE.md
HTMLParser defines a callback behavior that all parser implementations must adhere to, ensuring a consistent interface.
```elixir
@callback parse_document(binary(), Keyword.t()) :: {:ok, tree()} | {:error, String.t()}
```
--------------------------------
### Strategy Pattern Example
Source: https://github.com/philss/floki/blob/main/_autodocs/ARCHITECTURE.md
Text extraction employs a strategy pattern, dynamically selecting between Floki.DeepText and Floki.FlatText based on the `:deep` option.
```elixir
search_strategy = if opts[:deep], do: Floki.DeepText, else: Floki.FlatText
search_strategy.get(html, opts)
```
--------------------------------
### Example HTML Attributes Formats
Source: https://github.com/philss/floki/blob/main/_autodocs/types.md
Shows both the default list format and the map format for HTML attributes. The map format is used with the `attributes_as_maps` option.
```elixir
# List format (default)
[{"class", "box"}, {"id", "main"}]
# Map format (with attributes_as_maps option)
%{"class" => "box", "id" => "main"}
```
--------------------------------
### Example: Attribute selector for '[data-id='123']'
Source: https://github.com/philss/floki/blob/main/_autodocs/api-reference/selector.md
Shows the Floki.Selector.AttributeSelector struct created for an exact match on the 'data-id' attribute with the value '123'.
```elixir
# Selector for "[data-id='123']" creates:
%Selector{
attributes: [%Selector.AttributeSelector{
name: "data-id",
value: "123",
match_type: :exact
}]
}
```
--------------------------------
### Example HTML Node Types
Source: https://github.com/philss/floki/blob/main/_autodocs/types.md
Illustrates the different types of nodes that can exist in an HTML tree, including tag, text, comment, and doctype nodes.
```elixir
# Tag node
{"p", [{"class", "text"}], ["Content"]}
# Text node
"Hello, world!"
# Comment node
{:comment, "Note to developers"}
# Doctype node
{:doctype, "html", "", ""}
```
--------------------------------
### Flat Text Extraction Example
Source: https://github.com/philss/floki/blob/main/_autodocs/api-reference/text_extraction.md
Demonstrates extracting text only from direct children using `Floki.text/2` with `deep: false`, which utilizes FlatText.
```elixir
# With deep: false
html = [{"div", [], ["direct text", {"span", [], ["nested text"]}]}]
Floki.text(html, deep: false) # Uses FlatText internally
# => "direct text"
# Note: "nested text" from is NOT included
```
```elixir
# Another example
html = [{"article", [], [
"Lead paragraph. ",
{"p", [], ["First paragraph. "]},
{"p", [], ["Second paragraph. "]}
]}]
Floki.text(html, deep: false)
# => "Lead paragraph. "
```
--------------------------------
### Multiple Selectors
Source: https://github.com/philss/floki/blob/main/_autodocs/api-reference/css_selectors.md
Use a comma-separated list of selectors to match any elements that satisfy any of the individual selectors. Example: find 'h1', 'h2', or 'h3' elements.
```elixir
Floki.find(tree, "h1, h2, h3")
```
```elixir
Floki.find(tree, ".btn, button")
```
```elixir
Floki.find(tree, "#main, [role='main']")
```
--------------------------------
### Run Serialization Benchmark
Source: https://github.com/philss/floki/blob/main/_autodocs/ARCHITECTURE.md
Execute the serialization benchmark script to evaluate the performance of Floki's raw HTML serialization.
```bash
mix run benchs/raw_html.exs # Serialization benchmark
```
--------------------------------
### Floki Per-Function Options
Source: https://github.com/philss/floki/blob/main/_autodocs/REFERENCE.md
Demonstrates how to pass options to specific Floki functions for customization.
```elixir
# Parsing options
parse_document(html, attributes_as_maps: true, html_parser: Floki.HTMLParser.Html5ever)
# Text extraction options
text(tree, deep: false, sep: ", ", include_inputs: true)
# Raw HTML options
raw_html(tree, encode: false, pretty: true)
# Children options
children(node, include_text: false)
```
--------------------------------
### Configure Floki to Use html5ever Parser
Source: https://github.com/philss/floki/blob/main/README.md
Sets `html5ever` as the default HTML parser for Floki within the application configuration. This change takes effect after the application is compiled and started.
```elixir
# in config/config.exs
config :floki, :html_parser, Floki.HTMLParser.Html5ever
```
--------------------------------
### Query Operations: No Errors Returned
Source: https://github.com/philss/floki/blob/main/_autodocs/errors.md
Shows examples of Floki query functions (`find`, `attribute`, `text`, `get_by_id`, `children`) that do not raise errors or return error tuples, instead returning empty results or nil.
```elixir
# These never raise errors:
Floki.find(tree, "nonexistent") # => []
Floki.attribute(tree, "missing-attr") # => []
Floki.text([]) # => ""
Floki.get_by_id(tree, "no-such-id") # => nil
Floki.children({:comment, "text"}) # => nil
```
--------------------------------
### Basic Selector Usage
Source: https://github.com/philss/floki/blob/main/_autodocs/api-reference/css_selectors.md
Demonstrates how to pass CSS selectors as strings to Floki.find/2 for element selection.
```elixir
Floki.find(tree, "selector_expression")
```
--------------------------------
### attribute/2
Source: https://github.com/philss/floki/blob/main/_autodocs/api-reference/floki.md
Gets attribute values directly from already-selected elements.
```APIDOC
## attribute/2
### Description
Gets attribute values directly from already-selected elements.
### Method
N/A (Function Call)
### Parameters
#### Path Parameters
N/A
#### Query Parameters
N/A
#### Request Body
N/A
### Parameters
- **elements** (`html_tree()` | `html_node()`): Already-selected elements.
- **attribute_name** (`binary()`): Name of attribute to extract.
### Return
Returns list of attribute values.
### Request Example
```elixir
Floki.attribute([{"a", [{"href", "https://google.com"}], ["Google"]}], "href")
```
### Response
#### Success Response
List of attribute values.
#### Response Example
```elixir
["https://google.com"]
```
```
--------------------------------
### attribute/3
Source: https://github.com/philss/floki/blob/main/_autodocs/api-reference/floki.md
Gets attribute values of elements matching a selector within an HTML tree.
```APIDOC
## attribute/3
### Description
Gets attribute values of elements matching a selector.
### Method
N/A (Function Call)
### Parameters
#### Path Parameters
N/A
#### Query Parameters
N/A
#### Request Body
N/A
### Parameters
- **html** (`html_tree()` | `html_node()`): HTML tree to search.
- **selector** (`binary()`): CSS selector for matching elements.
- **attribute_name** (`binary()`): Name of attribute to extract.
### Return
Returns list of attribute values from matching elements.
### Request Example
```elixir
Floki.attribute([{"a", [{"href", "https://google.com"}], ["Google"]}], "a", "href")
```
### Response
#### Success Response
List of attribute values.
#### Response Example
```elixir
["https://google.com"]
```
```
--------------------------------
### Using Custom Selector Logic with Floki.find/2
Source: https://github.com/philss/floki/blob/main/_autodocs/api-reference/selector.md
Illustrates how to define a selector structure directly and pass it to `Floki.find/2` for matching. This is useful for building more complex or custom selection logic.
```elixir
html = [...]
selector = %Floki.Selector{type: "div", classes: ["alert"]}
selectors = [selector]
# Selectors can be passed directly to find
matches = Floki.find(html, selectors)
```
--------------------------------
### Run Selector Matching Benchmark
Source: https://github.com/philss/floki/blob/main/_autodocs/ARCHITECTURE.md
Execute the selector matching benchmark script to evaluate the performance of Floki's CSS selector matching.
```bash
mix run benchs/finder.exs # Selector matching benchmark
```
--------------------------------
### Floki Get Page Title
Source: https://github.com/philss/floki/blob/main/_autodocs/REFERENCE.md
Retrieves the text content of the title tag from an HTML document.
```elixir
{:ok, page} = Floki.parse_document(html)
page
|> Floki.find("title")
|> Floki.text()
```
--------------------------------
### Direct Selector Creation
Source: https://github.com/philss/floki/blob/main/_autodocs/api-reference/selector.md
Demonstrates how to programmatically create Floki selectors for use with `Floki.find/2`.
```APIDOC
## Usage Examples
### Direct Selector Creation
While selectors are usually created through parsing, you can create them directly:
```elixir
# Create a selector for ".container"
selector = %Floki.Selector{
classes: ["container"]
}
{:ok, html} = Floki.parse_fragment("")
Floki.find(html, selector)
```
```
--------------------------------
### Directly Create a Class Selector
Source: https://github.com/philss/floki/blob/main/_autodocs/api-reference/selector.md
Demonstrates direct creation of a selector for a CSS class. This is useful when you need to programmatically build selectors. It also shows how to parse HTML and find elements using this selector.
```elixir
# Create a selector for ".container"
selector = %Floki.Selector{
classes: ["container"]
}
{:ok, html} = Floki.parse_fragment("")
Floki.find(html, selector)
```
--------------------------------
### DeepText Module Behavior
Source: https://github.com/philss/floki/blob/main/_autodocs/api-reference/text_extraction.md
Defines the behavior for deep text extraction, recursively getting text from all descendants.
```elixir
defmodule Floki.DeepText do
def get(html_tree, opts :: Keyword.t()) :: binary()
end
```
--------------------------------
### Floki Project File Structure
Source: https://github.com/philss/floki/blob/main/_autodocs/REFERENCE.md
Illustrates the directory layout of the Floki Elixir project, showing the organization of documentation and API reference files.
```text
output/
├── REFERENCE.md (this file)
├── types.md (type definitions)
├── errors.md (error handling)
├── configuration.md (configuration options)
└── api-reference/
├── floki.md (main module)
├── html_parser.md (parser backends)
├── selector.md (CSS selector)
├── text_extraction.md (text utilities)
└── tree_manipulation.md (tree operations)
```
--------------------------------
### Get Only Immediate Text
Source: https://github.com/philss/floki/blob/main/_autodocs/api-reference/text_extraction.md
Use `Floki.text/2` with `deep: false` to extract text only from immediate children of the selected element.
```elixir
{:ok, page} = Floki.parse_document(html_string)
intro = page
|> Floki.find("article")
|> Floki.text(deep: false)
# => "Just the intro paragraph text"
```
--------------------------------
### Minimal Floki Configuration
Source: https://github.com/philss/floki/blob/main/_autodocs/configuration.md
Sets the default HTML parser to Mochiweb and enables raw HTML encoding. This is an optional minimal configuration.
```elixir
config :floki, :html_parser, Floki.HTMLParser.Mochiweb
config :floki, :encode_raw_html, true
```
--------------------------------
### Configure Mochiweb Parser
Source: https://github.com/philss/floki/blob/main/_autodocs/api-reference/html_parser.md
Configure Floki to use the Mochiweb parser, which is built-in and has faster startup but is not fully HTML5-compliant.
```elixir
config :floki, :html_parser, Floki.HTMLParser.Mochiweb
```
--------------------------------
### Dash-Prefixed Match Attribute Selector
Source: https://github.com/philss/floki/blob/main/_autodocs/api-reference/css_selectors.md
Matches elements where the attribute value is exactly equal or starts with the value followed by a dash.
```elixir
Floki.find(tree, "[lang|='en']")
```
--------------------------------
### Run Mix Tasks for Testing Parsers
Source: https://github.com/philss/floki/blob/main/_autodocs/configuration.md
Execute specific mix tasks to test Floki with different HTML parsers (Mochiweb, FastHtml, Html5ever) or all available parsers.
```bash
# Test with specific parser
mix test_mochiweb
mix test_fast_html
mix test_html5ever
# Test with all parsers
mix test_all
```
--------------------------------
### Benchmark HTML Parsing Performance
Source: https://github.com/philss/floki/blob/main/README.md
Commands to run performance benchmarks for Floki's HTML parsing capabilities. These scripts help in evaluating the speed of different parsing methods.
```sh
sh benchs/extract.sh
mix run benchs/parse_document.exs
```
--------------------------------
### Child Combinator (>)
Source: https://github.com/philss/floki/blob/main/_autodocs/api-reference/css_selectors.md
Matches elements that are direct children of a specified element. Example: find 'li' elements that are direct children of a 'ul'.
```elixir
Floki.find(tree, "ul > li")
```
```elixir
Floki.find(tree, "div > p")
```
--------------------------------
### Configure Floki HTML Parser for Production
Source: https://github.com/philss/floki/blob/main/_autodocs/configuration.md
Set the HTML parser to FastHtml for performance in production environments.
```elixir
# config/prod.exs
config :floki, :html_parser, Floki.HTMLParser.FastHtml
```
--------------------------------
### Run Parsing Benchmark
Source: https://github.com/philss/floki/blob/main/_autodocs/ARCHITECTURE.md
Execute the parsing benchmark script to evaluate the performance of Floki's parsing capabilities.
```bash
mix run benchs/parse_document.exs # Parsing benchmark
```
--------------------------------
### General Sibling Combinator (~)
Source: https://github.com/philss/floki/blob/main/_autodocs/api-reference/css_selectors.md
Matches elements that follow (are siblings of) a specified element. Example: find 'p' elements that are siblings following an 'h1'.
```elixir
Floki.find(tree, "h1 ~ p")
```
```elixir
Floki.find(tree, ".active ~ .item")
```
--------------------------------
### Raw HTML Encoding Behavior
Source: https://github.com/philss/floki/blob/main/_autodocs/configuration.md
Demonstrates the behavior of `raw_html/2` with and without the `encode` option. Shows how special characters are handled.
```elixir
# With encode: true (default)
Floki.raw_html({"div", [], ["5 > 3"]})
# => "5 > 3
"
# With encode: false
Floki.raw_html({"div", [], ["5 > 3"]}, encode: false)
# => "5 > 3
"
```
--------------------------------
### Descendant Combinator (space)
Source: https://github.com/philss/floki/blob/main/_autodocs/api-reference/css_selectors.md
Matches elements that are descendants of a specified element, regardless of depth. Example: find all 'p' tags within a 'div'.
```elixir
Floki.find(tree, "div p")
```
```elixir
Floki.find(tree, "article .highlight")
```
--------------------------------
### Get Children Excluding Text Nodes
Source: https://github.com/philss/floki/blob/main/_autodocs/configuration.md
Retrieves the direct children of a node, excluding text nodes. Useful for iterating over element children only.
```elixir
Floki.children(node, include_text: false)
```
--------------------------------
### Get Element by ID with Special Characters
Source: https://github.com/philss/floki/blob/main/_autodocs/api-reference/css_selectors.md
Use Floki.get_by_id/2 for selecting elements with IDs containing special characters that require CSS escaping.
```elixir
# For ID with special characters
Floki.get_by_id(tree, "my-special:id")
```
--------------------------------
### Suppressing Logger Messages
Source: https://github.com/philss/floki/blob/main/_autodocs/errors.md
Provides configuration for suppressing debug log messages from Floki at compile time by filtering messages starting with 'Floki'.
```elixir
# config/config.exs
config :logger, compile_time_purge_matching: [
{Logger, :debug, ["Floki" <> _]}
]
```
--------------------------------
### Safe Error Pattern: Parser Selection with Fallback
Source: https://github.com/philss/floki/blob/main/_autodocs/errors.md
Illustrates selecting different parsers, attempting a strict parser first and falling back to a more lenient one if an error occurs.
```elixir
def parse_with_fallback(html) do
# Try strict parser first
case Floki.parse_document(html, html_parser: Floki.HTMLParser.Html5ever) do
{:ok, tree} ->
{:ok, tree, "Html5ever"}
{:error, _} ->
# Fallback to lenient parser
case Floki.parse_document(html, html_parser: Floki.HTMLParser.Mochiweb) do
{:ok, tree} -> {:ok, tree, "Mochiweb"}
{:error, msg} -> {:error, msg}
end
end
end
```
--------------------------------
### Try-Rescue for Bang Functions
Source: https://github.com/philss/floki/blob/main/_autodocs/errors.md
Illustrates using `try/rescue` when dealing with bang versions of parsing functions, suitable when input is expected to be well-formed or immediate failure is desired.
```elixir
try do
tree = Floki.parse_document!(html)
Floki.find(tree, "p")
rescue
e in Floki.ParseError ->
IO.puts("Parse failed: #{e.message}")
end
```
--------------------------------
### Implement a Custom HTML Parser
Source: https://github.com/philss/floki/blob/main/_autodocs/api-reference/html_parser.md
Create a custom parser module by implementing the `Floki.HTMLParser` behavior. This includes defining `parse_document`, `parse_fragment`, and optionally `parse_document_with_attributes_as_maps` and `parse_fragment_with_attributes_as_maps`.
```elixir
defmodule MyCustomParser do
@behaviour Floki.HTMLParser
@impl true
def parse_document(html, args) do
# Your implementation
{:ok, parsed_tree}
end
@impl true
def parse_fragment(html, args) do
# Your implementation
{:ok, parsed_tree}
end
@impl true
def parse_document_with_attributes_as_maps(html, args) do
# Optional: implement if your parser supports it
{:error, "Not implemented"}
end
@impl true
def parse_fragment_with_attributes_as_maps(html, args) do
# Optional: implement if your parser supports it
{:error, "Not implemented"}
end
end
# Then configure it
config :floki, :html_parser, MyCustomParser
```
--------------------------------
### Add html5ever Dependency for Alternative Parsing
Source: https://github.com/philss/floki/blob/main/README.md
Includes `html5ever` as a dependency in `mix.exs` to use it as an alternative HTML parser with Floki. This requires running `mix deps.get` and `mix compile`.
```elixir
defp deps do
[
{:floki, "~> 0.38.0"},
{:html5ever, "~> 0.16.0"}
]
end
```
--------------------------------
### Get Child Elements
Source: https://github.com/philss/floki/blob/main/_autodocs/REFERENCE.md
Retrieve direct child nodes of a given node using `Floki.children`. The `include_text` option can be used to exclude text nodes.
```elixir
# Get child elements
Floki.children(node)
Floki.children(node, include_text: false)
```
--------------------------------
### Supported Pseudo-Classes
Source: https://github.com/philss/floki/blob/main/_autodocs/api-reference/selector.md
A list of pseudo-classes supported by Floki for advanced element selection.
```APIDOC
## Supported Pseudo-Classes
- `:first-child` - First child of parent
- `:last-child` - Last child of parent
- `:nth-child(n)` - nth child (value is the `n` expression)
- `:nth-last-child(n)` - nth from last
- `:first-of-type` - First of its type
- `:last-of-type` - Last of its type
- `:nth-of-type(n)` - nth of its type
- `:nth-last-of-type(n)` - nth from last of its type
- `:checked` - Checked checkbox/radio/option
- `:disabled` - Disabled form element
- `:not(selector)` - Negation (value is a selector)
- `:has(selector)` - Has child matching selector (value is a selector)
- `:fl-contains(text)` - Contains text (Floki custom, case-sensitive)
- `:fl-icontains(text)` - Contains text (Floki custom, case-insensitive)
**Example:**
```elixir
# Selector for "p:first-child" creates:
%Selector{
type: "p",
pseudo_classes: [%Selector.PseudoClass{name: "first-child", value: nil}]
}
```
```
--------------------------------
### Adjacent Sibling Combinator (+)
Source: https://github.com/philss/floki/blob/main/_autodocs/api-reference/css_selectors.md
Matches elements that immediately follow (are adjacent siblings of) a specified element. Example: find 'p' elements immediately following an 'h1'.
```elixir
Floki.find(tree, "h1 + p")
```
```elixir
Floki.find(tree, ".active + .item")
```
--------------------------------
### Recommended Tuple-based Error Handling
Source: https://github.com/philss/floki/blob/main/_autodocs/errors.md
Shows the recommended pattern for handling potential parsing errors using `case` statements with `{:ok, tree}` and `{:error, reason}` tuples.
```elixir
case Floki.parse_document(html) do
{:ok, tree} ->
# Process tree
Floki.find(tree, "p")
{:error, reason} ->
# Handle error
IO.puts("Parse failed: #{reason}")
end
```
--------------------------------
### Get Attribute Values from Selected Elements
Source: https://github.com/philss/floki/blob/main/_autodocs/api-reference/floki.md
Retrieves attribute values directly from a list of already selected HTML elements. Useful when you have specific nodes and need their attributes.
```elixir
Floki.attribute([{"a", [{"href", "https://google.com"}], ["Google"]}], "href")
# => ["https://google.com"]
```
```elixir
Floki.attribute([{"a", [{"href", "https://google.com"}, {"data-name", "google"}], ["Google"]}], "data-name")
# => ["google"]
```
--------------------------------
### Render HTML with Options
Source: https://github.com/philss/floki/blob/main/_autodocs/configuration.md
Renders an HTML tree to a string, with options for disabling entity encoding and enabling pretty-printing for better readability.
```elixir
html_string = Floki.raw_html(tree, encode: false, pretty: true)
```
--------------------------------
### Get Element by ID in Floki
Source: https://github.com/philss/floki/blob/main/_autodocs/api-reference/floki.md
Finds the first HTML node with a specific ID. Handles special characters in IDs effectively. Returns nil if no match is found.
```elixir
html = "hello"
{:ok, html_tree} = Floki.parse_fragment(html)
Floki.get_by_id(html_tree, "id?foo_special:chars")
# => {"span", [{"id", "id?foo_special:chars"}], ["hello"]}
Floki.get_by_id(html_tree, "does-not-exist")
# => nil
```
--------------------------------
### Configure Floki HTML Parser for Testing
Source: https://github.com/philss/floki/blob/main/_autodocs/configuration.md
Configure Floki to use the FastHtml parser specifically for testing purposes.
```elixir
# config/test.exs
config :floki, :html_parser, Floki.HTMLParser.FastHtml
```
--------------------------------
### Get Text from Element
Source: https://github.com/philss/floki/blob/main/README.md
Use `Floki.text/1` to extract the text content from a list of Floki nodes. This function is useful for retrieving human-readable text from specific parts of an HTML document.
```elixir
document
|> Floki.find(".headline")
|> Floki.text()
# => "Floki"
```
--------------------------------
### Finding Nodes and Returning Tree Parts
Source: https://github.com/philss/floki/blob/main/_autodocs/types.md
Demonstrates how the `Floki.find/2` function searches an HTML tree for nodes matching a CSS selector and returns the matching nodes as part of an `html_tree`.
```elixir
matches = Floki.find(tree, "p") # Returns [html_node(), ...]
```
--------------------------------
### Complex Selector Combinations
Source: https://github.com/philss/floki/blob/main/_autodocs/api-reference/css_selectors.md
Combine multiple selectors and features for precise element selection. Examples include class selectors, child combinators, disabled inputs, and elements with specific attributes.
```elixir
# Button with class, direct child of container
Floki.find(tree, ".container > button.primary")
```
```elixir
# Links that are adjacent to headings
Floki.find(tree, "h2 + p a")
```
```elixir
# Disabled inputs that are children of form
Floki.find(tree, "form input:disabled")
```
```elixir
# Paragraphs without intro class, in articles
Floki.find(tree, "article > p:not(.intro)")
```
```elixir
# Images in links that have alt text
Floki.find(tree, "a[href] img[alt]")
```
```elixir
# All elements with data attributes
Floki.find(tree, "[data-toggle], [data-id], [data-test]")
```
```elixir
# List items that are first of their type with class
Floki.find(tree, "li.active:first-of-type")
```
```elixir
# Divs containing buttons
Floki.find(tree, "div:has(button)")
```
```elixir
# Text matching
Floki.find(tree, "p:fl-contains('Copyright')")
```
--------------------------------
### Get Attribute Values by CSS Selector
Source: https://github.com/philss/floki/blob/main/_autodocs/api-reference/floki.md
Extracts attribute values from HTML elements that match a given CSS selector. Ensure the HTML tree and attribute name are correctly provided.
```elixir
Floki.attribute([{"a", [{"href", "https://google.com"}], ["Google"]}], "a", "href")
# => ["https://google.com"]
```
```elixir
Floki.attribute([{"a", [{"href", "https://e.corp.com"}, {"data-name", "e.corp"}], ["E.Corp"]}], "a[data-name]", "data-name")
# => ["e.corp"]
```
--------------------------------
### Configure HTML Parser Backend
Source: https://github.com/philss/floki/blob/main/_autodocs/configuration.md
Sets the default HTML parser backend for all parsing operations. Requires the specified parser module to be available.
```elixir
config :floki, :html_parser, Floki.HTMLParser.FastHtml
```
--------------------------------
### Get Direct Child Nodes of an HTML Node
Source: https://github.com/philss/floki/blob/main/_autodocs/api-reference/floki.md
The children function retrieves the direct child nodes of a given HTML node. By default, it includes text nodes, but this can be controlled with the :include_text option.
```elixir
Floki.children({"div", [], ["text", {"span", [], []}]})
# => ["text", {"span", [], []}]
```
```elixir
Floki.children({"div", [], ["text", {"span", [], []}]}, include_text: false)
# => [{"span", [], []}]
```
```elixir
Floki.children({:comment, "comment"})
# => nil
```
--------------------------------
### Configure Html5ever Parser
Source: https://github.com/philss/floki/blob/main/_autodocs/api-reference/html_parser.md
Configure Floki to use the Html5ever parser, which is fully HTML5 compliant and based on Rust. Precompiled binaries are available.
```elixir
config :floki, :html_parser, Floki.HTMLParser.Html5ever
```
--------------------------------
### Get Attribute from Existing Nodes
Source: https://github.com/philss/floki/blob/main/README.md
Use `Floki.attribute/2` to fetch an attribute's value from a list of existing Floki nodes. This is useful when you already have the nodes and want to extract specific attribute data.
```elixir
document
|> Floki.find(".example")
|> Floki.attribute("class")
# => ["example"]
```
--------------------------------
### Floki.text/2 Implementation
Source: https://github.com/philss/floki/blob/main/_autodocs/api-reference/text_extraction.md
Shows the internal implementation of `Floki.text/2`, which selects between DeepText and FlatText based on the `:deep` option.
```elixir
def text(html, opts \\\[]) do
defaults = [deep: true, js: false, style: true, sep: "", include_inputs: false]
opts = Keyword.validate!(opts, defaults)
search_strategy = if opts[:deep], do: Floki.DeepText, else: Floki.FlatText
search_strategy.get(html, opts)
end
```
--------------------------------
### Custom Selector Logic
Source: https://github.com/philss/floki/blob/main/_autodocs/api-reference/selector.md
Illustrates how to combine custom selector logic with `Floki.find/2` for flexible HTML querying.
```APIDOC
### Custom Selector Logic
While `Floki.find/2` covers most use cases, you can build custom logic with selectors:
```elixir
html = [...] # Assume html is populated
selector = %Floki.Selector{type: "div", classes: ["alert"]}
selectors = [selector]
# Selectors can be passed directly to find
matches = Floki.find(html, selectors)
```
```
--------------------------------
### Floki.DeepText.get/2
Source: https://github.com/philss/floki/blob/main/_autodocs/api-reference/text_extraction.md
Gets all text from an HTML tree recursively, including text in deeply nested elements. It ignores comments, processing instructions, and by default, script and style tags. It handles
tags by inserting newlines and can include input/textarea values with the :include_inputs option.
```APIDOC
## Floki.DeepText.get/2
### Description
Recursively extracts all text content from an HTML tree, including text within deeply nested elements. This is the default behavior for `Floki.text/2` when the `:deep` option is true.
### Method
```elixir
def get(html_tree, opts :: Keyword.t()) :: binary()
```
### Parameters
#### Options (`opts`)
- **`:deep`** (boolean) - Implicitly true for this module. Not configurable.
- **`:js`** (boolean) - Optional - Defaults to `false`. Include text content from `