### Install Floki in Livebook or Script Source: https://github.com/philss/floki/blob/main/README.md Provides instructions for installing Floki using `Mix.install/2`, suitable for use in Livebook or standalone scripts. This method avoids modifying the `mix.exs` file. ```elixir Mix.install([ {:floki, "~> 0.38.0"} ]) ``` -------------------------------- ### Complete Floki Configuration Example Source: https://github.com/philss/floki/blob/main/_autodocs/configuration.md A comprehensive example of Floki configuration, setting the HTML parser, raw HTML encoding, and self-closing tags. Includes test-specific configuration for the parser. ```elixir # config/config.exs config :floki, html_parser: Floki.HTMLParser.Html5ever, encode_raw_html: true, self_closing_tags: [ "area", "base", "br", "col", "embed", "hr", "img", "input", "keygen", "link", "meta", "param", "source", "track", "wbr" ] # config/test.exs (test-specific) import Config config :floki, html_parser: Floki.HTMLParser.Mochiweb ``` -------------------------------- ### Example HTML Comments Source: https://github.com/philss/floki/blob/main/_autodocs/types.md Shows examples of HTML comment nodes. ```elixir {:comment, " This is a comment "} {:comment, " TODO: implement pagination "} ``` -------------------------------- ### Floki Global Configuration Source: https://github.com/philss/floki/blob/main/_autodocs/REFERENCE.md Examples of global configuration settings for Floki in config/config.exs. ```elixir # Choose HTML parser config :floki, :html_parser, Floki.HTMLParser.FastHtml # Control encoding in raw_html config :floki, :encode_raw_html, true # Customize self-closing tags config :floki, :self_closing_tags, ["br", "hr", ...] ``` -------------------------------- ### Example HTML Doctype Declarations Source: https://github.com/philss/floki/blob/main/_autodocs/types.md Provides examples of DOCTYPE declarations, including HTML5 and XHTML formats. ```elixir {:doctype, "html", "", ""} # HTML5 {:doctype, "html", "-//W3C//DTD XHTML 1.0 Transitional//EN", "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"} ``` -------------------------------- ### Example HTML Tag Structures Source: https://github.com/philss/floki/blob/main/_autodocs/types.md Shows various examples of HTML tag structures, including elements with attributes, nested elements, and empty elements. ```elixir {"div", [{"id", "content"}], ["Hello"]} {"p", [{"class", "intro"}], ["Welcome"]} {"a", [{"href", "http://example.com"}], ["Link"]} {"div", [], []} # Empty div {"ul", [], [{"li", [], ["Item 1"]}, {"li", [], ["Item 2"]}]} ``` -------------------------------- ### Example HTML Declarations Source: https://github.com/philss/floki/blob/main/_autodocs/types.md Illustrates the structure of XML processing instructions, including the target and attributes. ```elixir {:pi, "xml", [{"version", "1.0"}, {"encoding", "utf-8"}]} {:pi, "xml-stylesheet", [{"href", "style.xsl"}]} ``` -------------------------------- ### Example HTML Attributes Map Source: https://github.com/philss/floki/blob/main/_autodocs/types.md Provides an example of an HTML attributes map. ```elixir %{"class" => "alert warning", "data-level" => "high"} ``` -------------------------------- ### Example HTML Attributes Source: https://github.com/philss/floki/blob/main/_autodocs/types.md Illustrates the string tuple format for HTML attributes. ```elixir {"class", "container"} {"id", "main"} {"data-count", "42"} ``` -------------------------------- ### Example HTML Text Nodes Source: https://github.com/philss/floki/blob/main/_autodocs/types.md Illustrates plain text content nodes, including whitespace. ```elixir "Hello, World!" "Some text content" " " # Whitespace text node ``` -------------------------------- ### HTML Tree Examples Source: https://github.com/philss/floki/blob/main/_autodocs/types.md Illustrates different forms of an HTML tree, including a parsed document, a parsed fragment, and an empty tree. ```elixir # Parsed document [{"html", [], [{"head", [], []}, {"body", [], ["Content"]}]}] ``` ```elixir # Parsed fragment [{"p", [], ["Hello"]}, {"p", [], ["World"]}] ``` ```elixir # Empty tree [] ``` -------------------------------- ### Floki Combinators Source: https://github.com/philss/floki/blob/main/_autodocs/REFERENCE.md Examples of using CSS combinators with Floki's find function. ```elixir find(tree, "div p") # Descendant find(tree, "div > p") # Child find(tree, "p + p") # Adjacent sibling find(tree, "p ~ p") # General sibling ``` -------------------------------- ### Parse Document with Mochiweb Source: https://github.com/philss/floki/blob/main/_autodocs/api-reference/html_parser.md Example of parsing an HTML document using the Mochiweb parser explicitly. ```elixir Floki.parse_document(html, html_parser: Floki.HTMLParser.Mochiweb) ``` -------------------------------- ### Floki Pseudo-Classes Source: https://github.com/philss/floki/blob/main/_autodocs/REFERENCE.md Examples of using pseudo-classes with Floki's find function for selecting specific elements. ```elixir find(tree, ":first-child") # First child find(tree, ":last-child") # Last child find(tree, ":nth-child(2)") # nth child find(tree, ":nth-last-child(2)") # nth from last find(tree, ":first-of-type") # First of type find(tree, ":last-of-type") # Last of type find(tree, ":nth-of-type(2)") # nth of type find(tree, ":checked") # Checked input find(tree, ":disabled") # Disabled element find(tree, ":not(p)") # Negation find(tree, ":has(a)") # Has child find(tree, ":root") # Root element find(tree, ":fl-contains('text'" find(tree, ":fl-icontains('foo'" ``` -------------------------------- ### Parse Document with FastHtml Source: https://github.com/philss/floki/blob/main/_autodocs/api-reference/html_parser.md Example of parsing an HTML document using the FastHtml parser explicitly. ```elixir Floki.parse_document(html, html_parser: Floki.HTMLParser.FastHtml) ``` -------------------------------- ### CSS Selector Examples Source: https://github.com/philss/floki/blob/main/_autodocs/types.md Demonstrates the three ways to specify a CSS selector: as a string, a single parsed struct, or a list of structs for multiple selectors. ```elixir # String selector "div.container > p" ``` ```elixir # Parsed selector struct %Floki.Selector{type: "div", classes: ["container"]} ``` ```elixir # List of selectors (comma-separated) [ %Floki.Selector{type: "div"}, %Floki.Selector{type: "p"} ] ``` -------------------------------- ### Example: Parsing multiple selectors Source: https://github.com/philss/floki/blob/main/_autodocs/api-reference/selector.md Shows how to parse a string containing multiple comma-separated selectors, resulting in a list of individual Floki.Selector structs. ```elixir selectors = Floki.Selector.Parser.parse(".warning, .error") # => [%Selector{classes: ["warning"]}, %Selector{classes: ["error"]}] ``` -------------------------------- ### Selector Optimization Examples Source: https://github.com/philss/floki/blob/main/_autodocs/api-reference/css_selectors.md Demonstrates optimized and less optimized selector patterns. More specific selectors and use of child combinators generally lead to better performance. ```elixir # Fast - specific selector Floki.find(tree, "article.post > p.intro") ``` ```elixir # Slower - too broad Floki.find(tree, "p") ``` ```elixir # Fast - use child combinator Floki.find(tree, "nav > ul > li > a") ``` ```elixir # Slower - unnecessary descendants Floki.find(tree, "nav ul li a") ``` -------------------------------- ### Floki Multiple Selectors Source: https://github.com/philss/floki/blob/main/_autodocs/REFERENCE.md Example of using multiple alternative selectors with Floki's find function. ```elixir find(tree, "h1, h2, h3") # Multiple alternatives ``` -------------------------------- ### Immutability Example Source: https://github.com/philss/floki/blob/main/_autodocs/ARCHITECTURE.md All Floki operations adhere to immutability, returning new tree structures without modifying the original. ```elixir # All return new tree, original unchanged Floki.find(tree, selector) Floki.traverse_and_update(tree, fun) Floki.filter_out(tree, selector) ``` -------------------------------- ### Example: Parsing a complex selector Source: https://github.com/philss/floki/blob/main/_autodocs/api-reference/selector.md Demonstrates parsing a selector string with element type, class, and a child combinator into a structured Floki.Selector representation. ```elixir selectors = Floki.Selector.Parser.parse("div.container > p.text") # => [%Floki.Selector{type: "div", classes: ["container"], # combinator: %Selector.Combinator{match_type: :child, # selector: %Selector{type: "p", classes: ["text"]}}}] ``` -------------------------------- ### Example: Combinator for 'div > p' Source: https://github.com/philss/floki/blob/main/_autodocs/api-reference/selector.md Illustrates the resulting Floki.Selector struct when parsing the 'div > p' selector, showing the child combinator and the nested selector. ```elixir # Selector for "div > p" creates: %Selector{ type: "div", combinator: %Selector.Combinator{ match_type: :child, selector: %Selector{type: "p"} } } ``` -------------------------------- ### Accumulator Pattern Example Source: https://github.com/philss/floki/blob/main/_autodocs/ARCHITECTURE.md Tree traversal functions like `traverse_and_update` support accumulators for stateful operations during the traversal. ```elixir traverse_and_update(tree, accumulator, fn node, acc -> {new_node, new_acc} end) ``` -------------------------------- ### Extract All Text from a Page Source: https://github.com/philss/floki/blob/main/_autodocs/api-reference/text_extraction.md Demonstrates a practical example of extracting all text content from a parsed HTML document using `Floki.text/2`. ```elixir {:ok, page} = Floki.parse_document(html_string) full_text = Floki.text(page) # => "all text from all elements" ``` -------------------------------- ### Self-Closing Tag Rendering Example Source: https://github.com/philss/floki/blob/main/_autodocs/configuration.md Illustrates the effect of self-closing tag configuration on the output of `raw_html/2`. Shows how configured self-closing tags are rendered. ```elixir Floki.raw_html({"br", [], []}) # => "
" (when "br" is self-closing) Floki.raw_html({"div", [], []}) # => "
" (not self-closing) ``` -------------------------------- ### Delegation Pattern Example Source: https://github.com/philss/floki/blob/main/_autodocs/ARCHITECTURE.md The main Floki module delegates common operations to specialized modules like HTMLParser, Finder, and RawHTML. ```elixir defdelegate parse_document(document, opts \ []), to: Floki.HTMLParser defdelegate find(tree, selector), to: Floki.Finder defdelegate raw_html(tree, opts \ []), to: Floki.RawHTML ``` -------------------------------- ### Mochiweb Parser Example (Lenient) Source: https://github.com/philss/floki/blob/main/_autodocs/errors.md Demonstrates that the default Mochiweb parser often succeeds even with unclosed HTML tags, showing its lenient nature. ```elixir Floki.parse_document!("") # Succeeds - unclosed body tag is accepted ``` -------------------------------- ### Selector for First Child Element Source: https://github.com/philss/floki/blob/main/_autodocs/api-reference/selector.md Example of how a CSS selector like 'p:first-child' is represented internally. ```elixir # Selector for "p:first-child" creates: %Selector{ type: "p", pseudo_classes: [%Selector.PseudoClass{name: "first-child", value: nil}] } ``` -------------------------------- ### Deep Text Extraction Example Source: https://github.com/philss/floki/blob/main/_autodocs/api-reference/text_extraction.md Demonstrates how to extract all text deeply from an HTML tree using Floki.text/2, which defaults to DeepText. ```elixir html = [{"div", [], [{"p", [], ["Hello"]}, " ", {"strong", [], ["World"]}]}] Floki.text(html) # Uses DeepText internally # => "Hello World" ``` ```elixir # With separator Floki.text(html, sep: " | ") # => "Hello | World" ``` -------------------------------- ### Behavior Pattern Example Source: https://github.com/philss/floki/blob/main/_autodocs/ARCHITECTURE.md HTMLParser defines a callback behavior that all parser implementations must adhere to, ensuring a consistent interface. ```elixir @callback parse_document(binary(), Keyword.t()) :: {:ok, tree()} | {:error, String.t()} ``` -------------------------------- ### Strategy Pattern Example Source: https://github.com/philss/floki/blob/main/_autodocs/ARCHITECTURE.md Text extraction employs a strategy pattern, dynamically selecting between Floki.DeepText and Floki.FlatText based on the `:deep` option. ```elixir search_strategy = if opts[:deep], do: Floki.DeepText, else: Floki.FlatText search_strategy.get(html, opts) ``` -------------------------------- ### Example HTML Attributes Formats Source: https://github.com/philss/floki/blob/main/_autodocs/types.md Shows both the default list format and the map format for HTML attributes. The map format is used with the `attributes_as_maps` option. ```elixir # List format (default) [{"class", "box"}, {"id", "main"}] # Map format (with attributes_as_maps option) %{"class" => "box", "id" => "main"} ``` -------------------------------- ### Example: Attribute selector for '[data-id='123']' Source: https://github.com/philss/floki/blob/main/_autodocs/api-reference/selector.md Shows the Floki.Selector.AttributeSelector struct created for an exact match on the 'data-id' attribute with the value '123'. ```elixir # Selector for "[data-id='123']" creates: %Selector{ attributes: [%Selector.AttributeSelector{ name: "data-id", value: "123", match_type: :exact }] } ``` -------------------------------- ### Example HTML Node Types Source: https://github.com/philss/floki/blob/main/_autodocs/types.md Illustrates the different types of nodes that can exist in an HTML tree, including tag, text, comment, and doctype nodes. ```elixir # Tag node {"p", [{"class", "text"}], ["Content"]} # Text node "Hello, world!" # Comment node {:comment, "Note to developers"} # Doctype node {:doctype, "html", "", ""} ``` -------------------------------- ### Flat Text Extraction Example Source: https://github.com/philss/floki/blob/main/_autodocs/api-reference/text_extraction.md Demonstrates extracting text only from direct children using `Floki.text/2` with `deep: false`, which utilizes FlatText. ```elixir # With deep: false html = [{"div", [], ["direct text", {"span", [], ["nested text"]}]}] Floki.text(html, deep: false) # Uses FlatText internally # => "direct text" # Note: "nested text" from is NOT included ``` ```elixir # Another example html = [{"article", [], [ "Lead paragraph. ", {"p", [], ["First paragraph. "]}, {"p", [], ["Second paragraph. "]} ]}] Floki.text(html, deep: false) # => "Lead paragraph. " ``` -------------------------------- ### Multiple Selectors Source: https://github.com/philss/floki/blob/main/_autodocs/api-reference/css_selectors.md Use a comma-separated list of selectors to match any elements that satisfy any of the individual selectors. Example: find 'h1', 'h2', or 'h3' elements. ```elixir Floki.find(tree, "h1, h2, h3") ``` ```elixir Floki.find(tree, ".btn, button") ``` ```elixir Floki.find(tree, "#main, [role='main']") ``` -------------------------------- ### Run Serialization Benchmark Source: https://github.com/philss/floki/blob/main/_autodocs/ARCHITECTURE.md Execute the serialization benchmark script to evaluate the performance of Floki's raw HTML serialization. ```bash mix run benchs/raw_html.exs # Serialization benchmark ``` -------------------------------- ### Floki Per-Function Options Source: https://github.com/philss/floki/blob/main/_autodocs/REFERENCE.md Demonstrates how to pass options to specific Floki functions for customization. ```elixir # Parsing options parse_document(html, attributes_as_maps: true, html_parser: Floki.HTMLParser.Html5ever) # Text extraction options text(tree, deep: false, sep: ", ", include_inputs: true) # Raw HTML options raw_html(tree, encode: false, pretty: true) # Children options children(node, include_text: false) ``` -------------------------------- ### Configure Floki to Use html5ever Parser Source: https://github.com/philss/floki/blob/main/README.md Sets `html5ever` as the default HTML parser for Floki within the application configuration. This change takes effect after the application is compiled and started. ```elixir # in config/config.exs config :floki, :html_parser, Floki.HTMLParser.Html5ever ``` -------------------------------- ### Query Operations: No Errors Returned Source: https://github.com/philss/floki/blob/main/_autodocs/errors.md Shows examples of Floki query functions (`find`, `attribute`, `text`, `get_by_id`, `children`) that do not raise errors or return error tuples, instead returning empty results or nil. ```elixir # These never raise errors: Floki.find(tree, "nonexistent") # => [] Floki.attribute(tree, "missing-attr") # => [] Floki.text([]) # => "" Floki.get_by_id(tree, "no-such-id") # => nil Floki.children({:comment, "text"}) # => nil ``` -------------------------------- ### Basic Selector Usage Source: https://github.com/philss/floki/blob/main/_autodocs/api-reference/css_selectors.md Demonstrates how to pass CSS selectors as strings to Floki.find/2 for element selection. ```elixir Floki.find(tree, "selector_expression") ``` -------------------------------- ### attribute/2 Source: https://github.com/philss/floki/blob/main/_autodocs/api-reference/floki.md Gets attribute values directly from already-selected elements. ```APIDOC ## attribute/2 ### Description Gets attribute values directly from already-selected elements. ### Method N/A (Function Call) ### Parameters #### Path Parameters N/A #### Query Parameters N/A #### Request Body N/A ### Parameters - **elements** (`html_tree()` | `html_node()`): Already-selected elements. - **attribute_name** (`binary()`): Name of attribute to extract. ### Return Returns list of attribute values. ### Request Example ```elixir Floki.attribute([{"a", [{"href", "https://google.com"}], ["Google"]}], "href") ``` ### Response #### Success Response List of attribute values. #### Response Example ```elixir ["https://google.com"] ``` ``` -------------------------------- ### attribute/3 Source: https://github.com/philss/floki/blob/main/_autodocs/api-reference/floki.md Gets attribute values of elements matching a selector within an HTML tree. ```APIDOC ## attribute/3 ### Description Gets attribute values of elements matching a selector. ### Method N/A (Function Call) ### Parameters #### Path Parameters N/A #### Query Parameters N/A #### Request Body N/A ### Parameters - **html** (`html_tree()` | `html_node()`): HTML tree to search. - **selector** (`binary()`): CSS selector for matching elements. - **attribute_name** (`binary()`): Name of attribute to extract. ### Return Returns list of attribute values from matching elements. ### Request Example ```elixir Floki.attribute([{"a", [{"href", "https://google.com"}], ["Google"]}], "a", "href") ``` ### Response #### Success Response List of attribute values. #### Response Example ```elixir ["https://google.com"] ``` ``` -------------------------------- ### Using Custom Selector Logic with Floki.find/2 Source: https://github.com/philss/floki/blob/main/_autodocs/api-reference/selector.md Illustrates how to define a selector structure directly and pass it to `Floki.find/2` for matching. This is useful for building more complex or custom selection logic. ```elixir html = [...] selector = %Floki.Selector{type: "div", classes: ["alert"]} selectors = [selector] # Selectors can be passed directly to find matches = Floki.find(html, selectors) ``` -------------------------------- ### Run Selector Matching Benchmark Source: https://github.com/philss/floki/blob/main/_autodocs/ARCHITECTURE.md Execute the selector matching benchmark script to evaluate the performance of Floki's CSS selector matching. ```bash mix run benchs/finder.exs # Selector matching benchmark ``` -------------------------------- ### Floki Get Page Title Source: https://github.com/philss/floki/blob/main/_autodocs/REFERENCE.md Retrieves the text content of the title tag from an HTML document. ```elixir {:ok, page} = Floki.parse_document(html) page |> Floki.find("title") |> Floki.text() ``` -------------------------------- ### Direct Selector Creation Source: https://github.com/philss/floki/blob/main/_autodocs/api-reference/selector.md Demonstrates how to programmatically create Floki selectors for use with `Floki.find/2`. ```APIDOC ## Usage Examples ### Direct Selector Creation While selectors are usually created through parsing, you can create them directly: ```elixir # Create a selector for ".container" selector = %Floki.Selector{ classes: ["container"] } {:ok, html} = Floki.parse_fragment("

Text

") Floki.find(html, selector) ``` ``` -------------------------------- ### Directly Create a Class Selector Source: https://github.com/philss/floki/blob/main/_autodocs/api-reference/selector.md Demonstrates direct creation of a selector for a CSS class. This is useful when you need to programmatically build selectors. It also shows how to parse HTML and find elements using this selector. ```elixir # Create a selector for ".container" selector = %Floki.Selector{ classes: ["container"] } {:ok, html} = Floki.parse_fragment("

Text

") Floki.find(html, selector) ``` -------------------------------- ### DeepText Module Behavior Source: https://github.com/philss/floki/blob/main/_autodocs/api-reference/text_extraction.md Defines the behavior for deep text extraction, recursively getting text from all descendants. ```elixir defmodule Floki.DeepText do def get(html_tree, opts :: Keyword.t()) :: binary() end ``` -------------------------------- ### Floki Project File Structure Source: https://github.com/philss/floki/blob/main/_autodocs/REFERENCE.md Illustrates the directory layout of the Floki Elixir project, showing the organization of documentation and API reference files. ```text output/ ├── REFERENCE.md (this file) ├── types.md (type definitions) ├── errors.md (error handling) ├── configuration.md (configuration options) └── api-reference/ ├── floki.md (main module) ├── html_parser.md (parser backends) ├── selector.md (CSS selector) ├── text_extraction.md (text utilities) └── tree_manipulation.md (tree operations) ``` -------------------------------- ### Get Only Immediate Text Source: https://github.com/philss/floki/blob/main/_autodocs/api-reference/text_extraction.md Use `Floki.text/2` with `deep: false` to extract text only from immediate children of the selected element. ```elixir {:ok, page} = Floki.parse_document(html_string) intro = page |> Floki.find("article") |> Floki.text(deep: false) # => "Just the intro paragraph text" ``` -------------------------------- ### Minimal Floki Configuration Source: https://github.com/philss/floki/blob/main/_autodocs/configuration.md Sets the default HTML parser to Mochiweb and enables raw HTML encoding. This is an optional minimal configuration. ```elixir config :floki, :html_parser, Floki.HTMLParser.Mochiweb config :floki, :encode_raw_html, true ``` -------------------------------- ### Configure Mochiweb Parser Source: https://github.com/philss/floki/blob/main/_autodocs/api-reference/html_parser.md Configure Floki to use the Mochiweb parser, which is built-in and has faster startup but is not fully HTML5-compliant. ```elixir config :floki, :html_parser, Floki.HTMLParser.Mochiweb ``` -------------------------------- ### Dash-Prefixed Match Attribute Selector Source: https://github.com/philss/floki/blob/main/_autodocs/api-reference/css_selectors.md Matches elements where the attribute value is exactly equal or starts with the value followed by a dash. ```elixir Floki.find(tree, "[lang|='en']") ``` -------------------------------- ### Run Mix Tasks for Testing Parsers Source: https://github.com/philss/floki/blob/main/_autodocs/configuration.md Execute specific mix tasks to test Floki with different HTML parsers (Mochiweb, FastHtml, Html5ever) or all available parsers. ```bash # Test with specific parser mix test_mochiweb mix test_fast_html mix test_html5ever # Test with all parsers mix test_all ``` -------------------------------- ### Benchmark HTML Parsing Performance Source: https://github.com/philss/floki/blob/main/README.md Commands to run performance benchmarks for Floki's HTML parsing capabilities. These scripts help in evaluating the speed of different parsing methods. ```sh sh benchs/extract.sh mix run benchs/parse_document.exs ``` -------------------------------- ### Child Combinator (>) Source: https://github.com/philss/floki/blob/main/_autodocs/api-reference/css_selectors.md Matches elements that are direct children of a specified element. Example: find 'li' elements that are direct children of a 'ul'. ```elixir Floki.find(tree, "ul > li") ``` ```elixir Floki.find(tree, "div > p") ``` -------------------------------- ### Configure Floki HTML Parser for Production Source: https://github.com/philss/floki/blob/main/_autodocs/configuration.md Set the HTML parser to FastHtml for performance in production environments. ```elixir # config/prod.exs config :floki, :html_parser, Floki.HTMLParser.FastHtml ``` -------------------------------- ### Run Parsing Benchmark Source: https://github.com/philss/floki/blob/main/_autodocs/ARCHITECTURE.md Execute the parsing benchmark script to evaluate the performance of Floki's parsing capabilities. ```bash mix run benchs/parse_document.exs # Parsing benchmark ``` -------------------------------- ### General Sibling Combinator (~) Source: https://github.com/philss/floki/blob/main/_autodocs/api-reference/css_selectors.md Matches elements that follow (are siblings of) a specified element. Example: find 'p' elements that are siblings following an 'h1'. ```elixir Floki.find(tree, "h1 ~ p") ``` ```elixir Floki.find(tree, ".active ~ .item") ``` -------------------------------- ### Raw HTML Encoding Behavior Source: https://github.com/philss/floki/blob/main/_autodocs/configuration.md Demonstrates the behavior of `raw_html/2` with and without the `encode` option. Shows how special characters are handled. ```elixir # With encode: true (default) Floki.raw_html({"div", [], ["5 > 3"]}) # => "
5 > 3
" # With encode: false Floki.raw_html({"div", [], ["5 > 3"]}, encode: false) # => "
5 > 3
" ``` -------------------------------- ### Descendant Combinator (space) Source: https://github.com/philss/floki/blob/main/_autodocs/api-reference/css_selectors.md Matches elements that are descendants of a specified element, regardless of depth. Example: find all 'p' tags within a 'div'. ```elixir Floki.find(tree, "div p") ``` ```elixir Floki.find(tree, "article .highlight") ``` -------------------------------- ### Get Children Excluding Text Nodes Source: https://github.com/philss/floki/blob/main/_autodocs/configuration.md Retrieves the direct children of a node, excluding text nodes. Useful for iterating over element children only. ```elixir Floki.children(node, include_text: false) ``` -------------------------------- ### Get Element by ID with Special Characters Source: https://github.com/philss/floki/blob/main/_autodocs/api-reference/css_selectors.md Use Floki.get_by_id/2 for selecting elements with IDs containing special characters that require CSS escaping. ```elixir # For ID with special characters Floki.get_by_id(tree, "my-special:id") ``` -------------------------------- ### Suppressing Logger Messages Source: https://github.com/philss/floki/blob/main/_autodocs/errors.md Provides configuration for suppressing debug log messages from Floki at compile time by filtering messages starting with 'Floki'. ```elixir # config/config.exs config :logger, compile_time_purge_matching: [ {Logger, :debug, ["Floki" <> _]} ] ``` -------------------------------- ### Safe Error Pattern: Parser Selection with Fallback Source: https://github.com/philss/floki/blob/main/_autodocs/errors.md Illustrates selecting different parsers, attempting a strict parser first and falling back to a more lenient one if an error occurs. ```elixir def parse_with_fallback(html) do # Try strict parser first case Floki.parse_document(html, html_parser: Floki.HTMLParser.Html5ever) do {:ok, tree} -> {:ok, tree, "Html5ever"} {:error, _} -> # Fallback to lenient parser case Floki.parse_document(html, html_parser: Floki.HTMLParser.Mochiweb) do {:ok, tree} -> {:ok, tree, "Mochiweb"} {:error, msg} -> {:error, msg} end end end ``` -------------------------------- ### Try-Rescue for Bang Functions Source: https://github.com/philss/floki/blob/main/_autodocs/errors.md Illustrates using `try/rescue` when dealing with bang versions of parsing functions, suitable when input is expected to be well-formed or immediate failure is desired. ```elixir try do tree = Floki.parse_document!(html) Floki.find(tree, "p") rescue e in Floki.ParseError -> IO.puts("Parse failed: #{e.message}") end ``` -------------------------------- ### Implement a Custom HTML Parser Source: https://github.com/philss/floki/blob/main/_autodocs/api-reference/html_parser.md Create a custom parser module by implementing the `Floki.HTMLParser` behavior. This includes defining `parse_document`, `parse_fragment`, and optionally `parse_document_with_attributes_as_maps` and `parse_fragment_with_attributes_as_maps`. ```elixir defmodule MyCustomParser do @behaviour Floki.HTMLParser @impl true def parse_document(html, args) do # Your implementation {:ok, parsed_tree} end @impl true def parse_fragment(html, args) do # Your implementation {:ok, parsed_tree} end @impl true def parse_document_with_attributes_as_maps(html, args) do # Optional: implement if your parser supports it {:error, "Not implemented"} end @impl true def parse_fragment_with_attributes_as_maps(html, args) do # Optional: implement if your parser supports it {:error, "Not implemented"} end end # Then configure it config :floki, :html_parser, MyCustomParser ``` -------------------------------- ### Add html5ever Dependency for Alternative Parsing Source: https://github.com/philss/floki/blob/main/README.md Includes `html5ever` as a dependency in `mix.exs` to use it as an alternative HTML parser with Floki. This requires running `mix deps.get` and `mix compile`. ```elixir defp deps do [ {:floki, "~> 0.38.0"}, {:html5ever, "~> 0.16.0"} ] end ``` -------------------------------- ### Get Child Elements Source: https://github.com/philss/floki/blob/main/_autodocs/REFERENCE.md Retrieve direct child nodes of a given node using `Floki.children`. The `include_text` option can be used to exclude text nodes. ```elixir # Get child elements Floki.children(node) Floki.children(node, include_text: false) ``` -------------------------------- ### Supported Pseudo-Classes Source: https://github.com/philss/floki/blob/main/_autodocs/api-reference/selector.md A list of pseudo-classes supported by Floki for advanced element selection. ```APIDOC ## Supported Pseudo-Classes - `:first-child` - First child of parent - `:last-child` - Last child of parent - `:nth-child(n)` - nth child (value is the `n` expression) - `:nth-last-child(n)` - nth from last - `:first-of-type` - First of its type - `:last-of-type` - Last of its type - `:nth-of-type(n)` - nth of its type - `:nth-last-of-type(n)` - nth from last of its type - `:checked` - Checked checkbox/radio/option - `:disabled` - Disabled form element - `:not(selector)` - Negation (value is a selector) - `:has(selector)` - Has child matching selector (value is a selector) - `:fl-contains(text)` - Contains text (Floki custom, case-sensitive) - `:fl-icontains(text)` - Contains text (Floki custom, case-insensitive) **Example:** ```elixir # Selector for "p:first-child" creates: %Selector{ type: "p", pseudo_classes: [%Selector.PseudoClass{name: "first-child", value: nil}] } ``` ``` -------------------------------- ### Adjacent Sibling Combinator (+) Source: https://github.com/philss/floki/blob/main/_autodocs/api-reference/css_selectors.md Matches elements that immediately follow (are adjacent siblings of) a specified element. Example: find 'p' elements immediately following an 'h1'. ```elixir Floki.find(tree, "h1 + p") ``` ```elixir Floki.find(tree, ".active + .item") ``` -------------------------------- ### Recommended Tuple-based Error Handling Source: https://github.com/philss/floki/blob/main/_autodocs/errors.md Shows the recommended pattern for handling potential parsing errors using `case` statements with `{:ok, tree}` and `{:error, reason}` tuples. ```elixir case Floki.parse_document(html) do {:ok, tree} -> # Process tree Floki.find(tree, "p") {:error, reason} -> # Handle error IO.puts("Parse failed: #{reason}") end ``` -------------------------------- ### Get Attribute Values from Selected Elements Source: https://github.com/philss/floki/blob/main/_autodocs/api-reference/floki.md Retrieves attribute values directly from a list of already selected HTML elements. Useful when you have specific nodes and need their attributes. ```elixir Floki.attribute([{"a", [{"href", "https://google.com"}], ["Google"]}], "href") # => ["https://google.com"] ``` ```elixir Floki.attribute([{"a", [{"href", "https://google.com"}, {"data-name", "google"}], ["Google"]}], "data-name") # => ["google"] ``` -------------------------------- ### Render HTML with Options Source: https://github.com/philss/floki/blob/main/_autodocs/configuration.md Renders an HTML tree to a string, with options for disabling entity encoding and enabling pretty-printing for better readability. ```elixir html_string = Floki.raw_html(tree, encode: false, pretty: true) ``` -------------------------------- ### Get Element by ID in Floki Source: https://github.com/philss/floki/blob/main/_autodocs/api-reference/floki.md Finds the first HTML node with a specific ID. Handles special characters in IDs effectively. Returns nil if no match is found. ```elixir html = "hello" {:ok, html_tree} = Floki.parse_fragment(html) Floki.get_by_id(html_tree, "id?foo_special:chars") # => {"span", [{"id", "id?foo_special:chars"}], ["hello"]} Floki.get_by_id(html_tree, "does-not-exist") # => nil ``` -------------------------------- ### Configure Floki HTML Parser for Testing Source: https://github.com/philss/floki/blob/main/_autodocs/configuration.md Configure Floki to use the FastHtml parser specifically for testing purposes. ```elixir # config/test.exs config :floki, :html_parser, Floki.HTMLParser.FastHtml ``` -------------------------------- ### Get Text from Element Source: https://github.com/philss/floki/blob/main/README.md Use `Floki.text/1` to extract the text content from a list of Floki nodes. This function is useful for retrieving human-readable text from specific parts of an HTML document. ```elixir document |> Floki.find(".headline") |> Floki.text() # => "Floki" ``` -------------------------------- ### Finding Nodes and Returning Tree Parts Source: https://github.com/philss/floki/blob/main/_autodocs/types.md Demonstrates how the `Floki.find/2` function searches an HTML tree for nodes matching a CSS selector and returns the matching nodes as part of an `html_tree`. ```elixir matches = Floki.find(tree, "p") # Returns [html_node(), ...] ``` -------------------------------- ### Complex Selector Combinations Source: https://github.com/philss/floki/blob/main/_autodocs/api-reference/css_selectors.md Combine multiple selectors and features for precise element selection. Examples include class selectors, child combinators, disabled inputs, and elements with specific attributes. ```elixir # Button with class, direct child of container Floki.find(tree, ".container > button.primary") ``` ```elixir # Links that are adjacent to headings Floki.find(tree, "h2 + p a") ``` ```elixir # Disabled inputs that are children of form Floki.find(tree, "form input:disabled") ``` ```elixir # Paragraphs without intro class, in articles Floki.find(tree, "article > p:not(.intro)") ``` ```elixir # Images in links that have alt text Floki.find(tree, "a[href] img[alt]") ``` ```elixir # All elements with data attributes Floki.find(tree, "[data-toggle], [data-id], [data-test]") ``` ```elixir # List items that are first of their type with class Floki.find(tree, "li.active:first-of-type") ``` ```elixir # Divs containing buttons Floki.find(tree, "div:has(button)") ``` ```elixir # Text matching Floki.find(tree, "p:fl-contains('Copyright')") ``` -------------------------------- ### Get Attribute Values by CSS Selector Source: https://github.com/philss/floki/blob/main/_autodocs/api-reference/floki.md Extracts attribute values from HTML elements that match a given CSS selector. Ensure the HTML tree and attribute name are correctly provided. ```elixir Floki.attribute([{"a", [{"href", "https://google.com"}], ["Google"]}], "a", "href") # => ["https://google.com"] ``` ```elixir Floki.attribute([{"a", [{"href", "https://e.corp.com"}, {"data-name", "e.corp"}], ["E.Corp"]}], "a[data-name]", "data-name") # => ["e.corp"] ``` -------------------------------- ### Configure HTML Parser Backend Source: https://github.com/philss/floki/blob/main/_autodocs/configuration.md Sets the default HTML parser backend for all parsing operations. Requires the specified parser module to be available. ```elixir config :floki, :html_parser, Floki.HTMLParser.FastHtml ``` -------------------------------- ### Get Direct Child Nodes of an HTML Node Source: https://github.com/philss/floki/blob/main/_autodocs/api-reference/floki.md The children function retrieves the direct child nodes of a given HTML node. By default, it includes text nodes, but this can be controlled with the :include_text option. ```elixir Floki.children({"div", [], ["text", {"span", [], []}]}) # => ["text", {"span", [], []}] ``` ```elixir Floki.children({"div", [], ["text", {"span", [], []}]}, include_text: false) # => [{"span", [], []}] ``` ```elixir Floki.children({:comment, "comment"}) # => nil ``` -------------------------------- ### Configure Html5ever Parser Source: https://github.com/philss/floki/blob/main/_autodocs/api-reference/html_parser.md Configure Floki to use the Html5ever parser, which is fully HTML5 compliant and based on Rust. Precompiled binaries are available. ```elixir config :floki, :html_parser, Floki.HTMLParser.Html5ever ``` -------------------------------- ### Get Attribute from Existing Nodes Source: https://github.com/philss/floki/blob/main/README.md Use `Floki.attribute/2` to fetch an attribute's value from a list of existing Floki nodes. This is useful when you already have the nodes and want to extract specific attribute data. ```elixir document |> Floki.find(".example") |> Floki.attribute("class") # => ["example"] ``` -------------------------------- ### Floki.text/2 Implementation Source: https://github.com/philss/floki/blob/main/_autodocs/api-reference/text_extraction.md Shows the internal implementation of `Floki.text/2`, which selects between DeepText and FlatText based on the `:deep` option. ```elixir def text(html, opts \\\[]) do defaults = [deep: true, js: false, style: true, sep: "", include_inputs: false] opts = Keyword.validate!(opts, defaults) search_strategy = if opts[:deep], do: Floki.DeepText, else: Floki.FlatText search_strategy.get(html, opts) end ``` -------------------------------- ### Custom Selector Logic Source: https://github.com/philss/floki/blob/main/_autodocs/api-reference/selector.md Illustrates how to combine custom selector logic with `Floki.find/2` for flexible HTML querying. ```APIDOC ### Custom Selector Logic While `Floki.find/2` covers most use cases, you can build custom logic with selectors: ```elixir html = [...] # Assume html is populated selector = %Floki.Selector{type: "div", classes: ["alert"]} selectors = [selector] # Selectors can be passed directly to find matches = Floki.find(html, selectors) ``` ``` -------------------------------- ### Floki.DeepText.get/2 Source: https://github.com/philss/floki/blob/main/_autodocs/api-reference/text_extraction.md Gets all text from an HTML tree recursively, including text in deeply nested elements. It ignores comments, processing instructions, and by default, script and style tags. It handles
tags by inserting newlines and can include input/textarea values with the :include_inputs option. ```APIDOC ## Floki.DeepText.get/2 ### Description Recursively extracts all text content from an HTML tree, including text within deeply nested elements. This is the default behavior for `Floki.text/2` when the `:deep` option is true. ### Method ```elixir def get(html_tree, opts :: Keyword.t()) :: binary() ``` ### Parameters #### Options (`opts`) - **`:deep`** (boolean) - Implicitly true for this module. Not configurable. - **`:js`** (boolean) - Optional - Defaults to `false`. Include text content from `