itemloaders

https://itemloaders.readthedocs.io/en/latest/
Admin
itemloaders is a Python library that provides a convenient mechanism for populating data records...
Tokens:19,345
Snippets:233
Trust Score:9.6
Update:4 months ago
Show doc for...
Tokens:
Raw
### Getting Started with Item Loaders

Source: https://itemloaders.readthedocs.io/en/latest/index.html

Example demonstrating the basic usage of ItemLoader, including adding data via XPath, CSS, and literal values, and loading the item.

```python
from itemloaders import ItemLoader
from parsel import Selector

html_data = '''
<!DOCTYPE html>
<html>
    <head>
        <title>Some random product page</title>
    </head>
    <body>
        <div class="product_name">Some random product page</div>
        <p id="price">$ 100.12</p>
    </body>
</html>
'''
l = ItemLoader(selector=Selector(html_data))
l.add_xpath('name', '//div[@class="product_name"]/text()')
l.add_xpath('name', '//div[@class="product_title"]/text()')
l.add_css('price', '#price::text')
l.add_value('last_updated', 'today') # you can also use literal values
item = l.load_item()
item
# {'name': ['Some random product page'], 'price': ['$ 100.12'], 'last_updated': ['today']}

```

--------------------------------

### TakeFirst processor example

Source: https://itemloaders.readthedocs.io/en/latest/built-in-processors.html

Example demonstrating the usage of the TakeFirst processor to get the first non-empty value from a list.

```python
from itemloaders.processors import TakeFirst
proc = TakeFirst()
proc(['', 'one', 'two', 'three'])
```

--------------------------------

### get_xpath examples

Source: https://itemloaders.readthedocs.io/en/latest/api-reference.html

Examples of using get_xpath with different HTML snippets.

```python
loader.get_xpath('//p[@class="product-name"]')
```

```python
loader.get_xpath('//p[@id="price"]', TakeFirst(), re='the price is (.*)')
```

--------------------------------

### Join processor examples

Source: https://itemloaders.readthedocs.io/en/latest/built-in-processors.html

Examples illustrating the Join processor with default and custom separators.

```python
from itemloaders.processors import Join
proc = Join()
proc(['one', 'two', 'three'])
```

```python
proc = Join('<br>')
proc(['one', 'two', 'three'])
```

--------------------------------

### Installation

Source: https://itemloaders.readthedocs.io/en/latest/_sources/index.rst.txt

Command to install the itemloaders package.

```bash
pip install itemloaders
```

--------------------------------

### add_xpath examples

Source: https://itemloaders.readthedocs.io/en/latest/api-reference.html

Shows how to add data to an ItemLoader using XPath selectors, with an example of using a regular expression.

```python
# HTML snippet: <p class="product-name">Color TV</p>
loader.add_xpath('name', '//p[@class="product-name"]')
# HTML snippet: <p id="price">the price is $1200</p>
loader.add_xpath('price', '//p[@id="price"]', re='the price is (.*)')
```

--------------------------------

### Compose processor example

Source: https://itemloaders.readthedocs.io/en/latest/built-in-processors.html

Example demonstrating the usage of the Compose processor to chain functions.

```python
from itemloaders.processors import Compose
proc = Compose(lambda v: v[0], str.upper)
proc(['hello', 'world'])
```

--------------------------------

### TakeFirst processor example

Source: https://itemloaders.readthedocs.io/en/latest/_modules/itemloaders/processors.html

Example usage of the TakeFirst processor.

```python
>>> from itemloaders.processors import TakeFirst
>>> proc = TakeFirst()
>>> proc(['', 'one', 'two', 'three'])
```

--------------------------------

### ProductLoader Example

Source: https://itemloaders.readthedocs.io/en/latest/declaring-loaders.html

An example demonstrating the declaration of an Item Loader with custom input and output processors for 'name' and 'price' fields, along with default processors.

```python
from itemloaders import ItemLoader
from itemloaders.processors import TakeFirst, MapCompose, Join

class ProductLoader(ItemLoader):

    default_output_processor = TakeFirst()

    name_in = MapCompose(str.title)
    name_out = Join()

    # using a built-in processor
    price_in = MapCompose(str.strip)

    # using a function
    def price_out(self, values):
        return float(values[0])

loader = ProductLoader()
loader.add_value('name', 'plasma TV')
loader.add_value('price', '999.98')
loader.load_item()
# {'name': 'Plasma Tv', 'price': 999.98}
```

--------------------------------

### get_css examples

Source: https://itemloaders.readthedocs.io/en/latest/api-reference.html

Illustrates how to extract data using CSS selectors, including using TakeFirst processor and a regular expression.

```python
# HTML snippet: <p class="product-name">Color TV</p>
loader.get_css('p.product-name')
# HTML snippet: <p id="price">the price is $1200</p>
loader.get_css('p#price', TakeFirst(), re='the price is (.*)')
```

--------------------------------

### Identity processor example

Source: https://itemloaders.readthedocs.io/en/latest/_modules/itemloaders/processors.html

Example usage of the Identity processor.

```python
>>> from itemloaders.processors import Identity
>>> proc = Identity()
>>> proc(['one', 'two', 'three'])
```

--------------------------------

### Join processor examples

Source: https://itemloaders.readthedocs.io/en/latest/_modules/itemloaders/processors.html

Examples of using the Join processor with default and custom separators.

```python
>>> from itemloaders.processors import Join
>>> proc = Join()
>>> proc(['one', 'two', 'three'])
>>> proc = Join('<br>')
>>> proc(['one', 'two', 'three'])
```

--------------------------------

### Example HTML structure

Source: https://itemloaders.readthedocs.io/en/latest/_sources/nested-loaders.rst.txt

An example of an HTML footer structure from which related values will be extracted.

```html
<footer>
    <a class="social" href="https://facebook.com/whatever">Like Us</a>
    <a class="social" href="https://twitter.com/whatever">Follow Us</a>
    <a class="email" href="mailto:whatever@example.com">Email Us</a>
</footer>
```

--------------------------------

### Install itemloaders

Source: https://itemloaders.readthedocs.io/en/latest/index.html

Command to install the itemloaders package using pip.

```bash
pip install itemloaders

```

--------------------------------

### get_value example

Source: https://itemloaders.readthedocs.io/en/latest/api-reference.html

Demonstrates processing a string value with processors and a regular expression.

```python
>>> from itemloaders import ItemLoader
>>> from itemloaders.processors import TakeFirst
>>> loader = ItemLoader()
>>> loader.get_value('name: foo', TakeFirst(), str.upper, re='name: (.+)')
'FOO'
```

--------------------------------

### Identity processor example

Source: https://itemloaders.readthedocs.io/en/latest/built-in-processors.html

Example showing the Identity processor, which returns values unchanged.

```python
from itemloaders.processors import Identity
proc = Identity()
proc(['one', 'two', 'three'])
```

--------------------------------

### ItemLoader get_value example

Source: https://itemloaders.readthedocs.io/en/latest/_modules/itemloaders.html

Example demonstrating the usage of get_value with regular expressions and processors.

```python
>>> loader = ItemLoader()
>>> loader.get_value('name: foo', TakeFirst(), str.upper, re='name: (.+)')
'FOO'
```

--------------------------------

### MapCompose processor example

Source: https://itemloaders.readthedocs.io/en/latest/built-in-processors.html

Example demonstrating the MapCompose processor for composing functions that operate on individual elements.

```python
def filter_world(x):
    return None if x == 'world' else x

from itemloaders.processors import MapCompose
proc = MapCompose(filter_world, str.upper)
proc(['hello', 'world', 'this', 'is', 'something'])
```

--------------------------------

### add_value examples

Source: https://itemloaders.readthedocs.io/en/latest/api-reference.html

Demonstrates how to add values to an ItemLoader, including using regular expressions for extraction.

```python
loader.add_value('name', 'Color TV')
loader.add_value('colours', ['white', 'blue'])
loader.add_value('length', '100')
loader.add_value('name', 'name: foo', TakeFirst(), re='name: (.+)')
loader.add_value(None, {'name': 'foo', 'sex': 'male'})
```

--------------------------------

### SelectJmes processor examples

Source: https://itemloaders.readthedocs.io/en/latest/_modules/itemloaders/processors.html

Examples of using the SelectJmes processor for direct use and with JSON.

```python
>>> from itemloaders.processors import SelectJmes, Compose, MapCompose
>>> proc = SelectJmes("foo") #for direct use on lists and dictionaries
>>> proc({'foo': 'bar'})
>>> proc({'foo': {'bar': 'baz'}})
```

```python
>>> import json
>>> proc_single_json_str = Compose(json.loads, SelectJmes("foo"))
>>> proc_single_json_str('{"foo": "bar"}')
>>> proc_json_list = Compose(json.loads, MapCompose(SelectJmes('foo')))
>>> proc_json_list('[{"foo":"bar"}, {"baz":"tar"}]')
```

--------------------------------

### get_jmes examples

Source: https://itemloaders.readthedocs.io/en/latest/api-reference.html

Provides examples of extracting data using JMESPath selectors, including using TakeFirst processor and a regular expression.

```python
# HTML snippet: {"name": "Color TV"}
loader.get_jmes('name')
# HTML snippet: {"price": the price is $1200"}
loader.get_jmes('price', TakeFirst(), re='the price is (.*)')
```

--------------------------------

### add_css Example

Source: https://itemloaders.readthedocs.io/en/latest/api-reference.html

Example of using add_css to extract data from HTML using CSS selectors.

```python
# HTML snippet: <p class="product-name">Color TV</p>
loader.add_css('name', 'p.product-name')
# HTML snippet: <p id="price">the price is $1200</p>
loader.add_css('price', 'p#price', re='the price is (.*)')

```

--------------------------------

### Working with JSON

Source: https://itemloaders.readthedocs.io/en/latest/built-in-processors.html

Examples of using Compose, SelectJmes, and MapCompose for processing JSON strings.

```python
import json
proc_single_json_str = Compose(json.loads, SelectJmes("foo"))
proc_single_json_str('{"foo": "bar"}')
proc_json_list = Compose(json.loads, MapCompose(SelectJmes('foo')))
proc_json_list('[{"foo":"bar"}, {"baz":"tar"}]')
```

--------------------------------

### Method Chaining Example

Source: https://itemloaders.readthedocs.io/en/latest/_sources/release-notes.rst.txt

Demonstrates method chaining for add_* and replace_* methods in ItemLoader.

```python
loader.add_xpath("name", "//body/text()").add_value("url", "http://example.com")
```

--------------------------------

### SelectJmes processor example

Source: https://itemloaders.readthedocs.io/en/latest/built-in-processors.html

Example of using the SelectJmes processor to query JSON-like input with a JMESPath expression.

```python
from itemloaders.processors import SelectJmes, Compose, MapCompose
proc = SelectJmes("foo") #for direct use on lists and dictionaries
proc({'foo': 'bar'})
```

--------------------------------

### add_jmes Example

Source: https://itemloaders.readthedocs.io/en/latest/api-reference.html

Example of using add_jmes to extract data from JSON-like structures using JMESPath selectors.

```python
# HTML snippet: {"name": "Color TV"}
loader.add_jmes('name')

```

--------------------------------

### Example parse_length function accepting loader_context

Source: https://itemloaders.readthedocs.io/en/latest/_sources/loaders-context.rst.txt

Demonstrates a processor function that accepts and utilizes the loader_context to retrieve configuration like 'unit'.

```python
def parse_length(text, loader_context):
    unit = loader_context.get('unit', 'm')
    # ... length parsing code goes here ...
    return parsed_length
```

--------------------------------

### Typical Item Loader Usage

Source: https://itemloaders.readthedocs.io/en/latest/_sources/index.rst.txt

Example demonstrating how to use ItemLoader to extract data from HTML using XPath and CSS selectors, and add literal values.

```python
from itemloaders import ItemLoader
from parsel import Selector

html_data = '''
<!DOCTYPE html>
<html>
    <head>
        <title>Some random product page</title>
    </head>
    <body>
        <div class="product_name">Some random product page</div>
        <p id="price">$ 100.12</p>
    </body>
</html>
'''
l = ItemLoader(selector=Selector(html_data))
l.add_xpath('name', '//div[@class="product_name"]/text()')
l.add_xpath('name', '//div[@class="product_title"]/text()')
l.add_css('price', '#price::text')
l.add_value('last_updated', 'today') # you can also use literal values
item = l.load_item()
item
# {'name': ['Some random product page'], 'price': ['$ 100.12'], 'last_updated': ['today']}
```

--------------------------------

### Extending ProductLoader to strip dashes

Source: https://itemloaders.readthedocs.io/en/latest/extending-loaders.html

This example demonstrates how to extend the default ProductLoader to create a SiteSpecificLoader that removes leading and trailing dashes from product names using a custom strip_dashes function and MapCompose.

```python
from itemloaders.processors import MapCompose
from myproject.loaders import ProductLoader

def strip_dashes(x):
    return x.strip('-')

class SiteSpecificLoader(ProductLoader):
    name_in = MapCompose(strip_dashes, ProductLoader.name_in)

```

--------------------------------

### Item Loader Field Processing Example

Source: https://itemloaders.readthedocs.io/en/latest/_sources/processors.rst.txt

Illustrates the sequence of calls to input and output processors for a specific field ('name') within an ItemLoader.

```python
l = ItemLoader(selector=some_selector)
l.add_xpath('name', xpath1) # (1)
l.add_xpath('name', xpath2) # (2)
l.add_css('name', css) # (3)
l.add_value('name', 'test') # (4)
return l.load_item() # (5)
```

--------------------------------

### _get_xpathvalues method

Source: https://itemloaders.readthedocs.io/en/latest/_modules/itemloaders.html

Internal helper to get all values from XPath selectors.

```python
    def _get_xpathvalues(self, xpaths: str | Iterable[str], **kw: Any) -> list[Any]:
        self._check_selector_method()
        assert self.selector is not None
        xpaths = arg_to_iter(xpaths)
        return flatten(self.selector.xpath(xpath, **kw).getall() for xpath in xpaths)
```

--------------------------------

### Stripping dashes from product names

Source: https://itemloaders.readthedocs.io/en/latest/_sources/extending-loaders.rst.txt

This example demonstrates how to extend the default ProductLoader to remove leading and trailing dashes from product names using MapCompose and a custom strip_dashes function.

```python
from itemloaders.processors import MapCompose
from myproject.loaders import ProductLoader

def strip_dashes(x):
    return x.strip('-')

class SiteSpecificLoader(ProductLoader):
    name_in = MapCompose(strip_dashes, ProductLoader.name_in)
```

--------------------------------

### load_item()

Source: https://itemloaders.readthedocs.io/en/latest/api-reference.html

Populate the item with the data collected so far, and return it. The data collected is first passed through the output processors to get the final value to assign to each item field.

```python
load_item() -> Any[source]
```

--------------------------------

### _get_cssvalues method

Source: https://itemloaders.readthedocs.io/en/latest/_modules/itemloaders.html

Internal helper to get all values from CSS selectors.

```python
    def _get_cssvalues(self, csss: str | Iterable[str]) -> list[Any]:
        self._check_selector_method()
        assert self.selector is not None
        csss = arg_to_iter(csss)
        return flatten(self.selector.css(css).getall() for css in csss)
```

--------------------------------

### Extending ProductLoader for XML data

Source: https://itemloaders.readthedocs.io/en/latest/extending-loaders.html

This example shows how to extend ProductLoader for XML data by creating an XmlProductLoader that uses MapCompose to apply a remove_cdata function, effectively removing CDATA occurrences from product names.

```python
from itemloaders.processors import MapCompose
from myproject.ItemLoaders import ProductLoader
from myproject.utils.xml import remove_cdata

class XmlProductLoader(ProductLoader):
    name_in = MapCompose(remove_cdata, ProductLoader.name_in)

```

--------------------------------

### Removing CDATA from XML product names

Source: https://itemloaders.readthedocs.io/en/latest/_sources/extending-loaders.rst.txt

This example shows how to extend ProductLoader to remove CDATA occurrences from product names when parsing XML data, utilizing MapCompose and a remove_cdata utility function.

```python
from itemloaders.processors import MapCompose
from myproject.ItemLoaders import ProductLoader
from myproject.utils.xml import remove_cdata

class XmlProductLoader(ProductLoader):
    name_in = MapCompose(remove_cdata, ProductLoader.name_in)
```

--------------------------------

### ItemLoader _get_item_field_attr method

Source: https://itemloaders.readthedocs.io/en/latest/_modules/itemloaders.html

Helper method to get field metadata attributes from the ItemAdapter.

```python
def _get_item_field_attr(
    self, field_name: str, key: Any, default: Any = None
) -> Any:
    field_meta = ItemAdapter(self.item).get_field_meta(field_name)
    return field_meta.get(key, default)
```

--------------------------------

### Setting Item Loader context on instantiation

Source: https://itemloaders.readthedocs.io/en/latest/_sources/loaders-context.rst.txt

Illustrates passing context values as keyword arguments during Item Loader instantiation.

```python
loader = ItemLoader(product, unit='cm')
```

--------------------------------

### Setting Item Loader context on declaration with MapCompose

Source: https://itemloaders.readthedocs.io/en/latest/_sources/loaders-context.rst.txt

Demonstrates how to pass context values when declaring an Item Loader, specifically when using MapCompose.

```python
class ProductLoader(ItemLoader):
    length_out = MapCompose(parse_length, unit='cm')
```

--------------------------------

### Extracting values with nested loaders

Source: https://itemloaders.readthedocs.io/en/latest/_sources/nested-loaders.rst.txt

Demonstrates extracting social and email links from the footer using a nested loader, simplifying the XPath selectors.

```python
loader = ItemLoader()
# load stuff not in the footer
footer_loader = loader.nested_xpath('//footer')
footer_loader.add_xpath('social', 'a[@class = "social"]/@href')
footer_loader.add_xpath('email', 'a[@class = "email"]/@href')
# no need to call footer_loader.load_item()
loader.load_item()
```

--------------------------------

### Dataclass Alternative for Item Instantiation

Source: https://itemloaders.readthedocs.io/en/latest/api-reference.html

An alternative way to define an Item class using dataclasses when the default_item_class needs to be callable without arguments.

```python
from dataclasses import dataclass, field
from typing import Optional

@dataclass
class Product:
    name: Optional[str] = field(default=None)
    price: Optional[float] = field(default=None)

```

--------------------------------

### Join processor implementation

Source: https://itemloaders.readthedocs.io/en/latest/_modules/itemloaders/processors.html

The __init__ and __call__ methods for the Join processor.

```python
def __init__(self, separator: str = " "):
        self.separator = separator

    def __call__(self, values: Any) -> str:
        return self.separator.join(values)
```

--------------------------------

### SelectJmes processor implementation

Source: https://itemloaders.readthedocs.io/en/latest/_modules/itemloaders/processors.html

The __init__ and __call__ methods for the SelectJmes processor.

```python
def __init__(self, json_path: str):
        self.json_path: str = json_path
        import jmespath.parser  # noqa: PLC0415

        self.compiled_path: jmespath.parser.ParsedResult = jmespath.compile(
            self.json_path
        )

    def __call__(self, value: Any) -> Any:
        """Query value for the JMESPath query and return answer
        :param value: a data structure (dict, list) to extract from
        :return: Element extracted according to JMESPath query
        """
        return self.compiled_path.search(value)
```

--------------------------------

### Extracting values without nested loaders

Source: https://itemloaders.readthedocs.io/en/latest/_sources/nested-loaders.rst.txt

Demonstrates extracting social and email links from the footer using full XPath selectors without nested loaders.

```python
loader = ItemLoader()
# load stuff not in the footer
loader.add_xpath('social', '//footer/a[@class = "social"]/@href')
loader.add_xpath('email', '//footer/a[@class = "email"]/@href')
loader.load_item()
```

--------------------------------

### MapCompose Implementation

Source: https://itemloaders.readthedocs.io/en/latest/_modules/itemloaders/processors.html

The implementation of the MapCompose processor, showing how it iterates through functions and values.

```python
from __future__ import annotations

from collections import ChainMap
from typing import TYPE_CHECKING, Any

from itemloaders.common import wrap_loader_context
from itemloaders.utils import arg_to_iter

if TYPE_CHECKING:
    from collections.abc import Callable, Iterable, MutableMapping




class MapCompose:
    """
    A processor which is constructed from the composition of the given
    functions, similar to the :class:`Compose` processor. The difference with
    this processor is the way internal results are passed among functions,
    which is as follows:

    The input value of this processor is *iterated* and the first function is
    applied to each element. The results of these function calls (one for each element)
    are concatenated to construct a new iterable, which is then used to apply the
    second function, and so on, until the last function is applied to each
    value of the list of values collected so far. The output values of the last
    function are concatenated together to produce the output of this processor.

    Each particular function can return a value or a list of values, which is
    flattened with the list of values returned by the same function applied to
    the other input values. The functions can also return ``None`` in which
    case the output of that function is ignored for further processing over the
    chain.

    This processor provides a convenient way to compose functions that only
    work with single values (instead of iterables). For this reason the
    :class:`MapCompose` processor is typically used as input processor, since
    data is often extracted using the
    :meth:`~parsel.selector.Selector.extract` method of `parsel selectors`_,
    which returns a list of unicode strings.

    The example below should clarify how it works:

    >>> def filter_world(x):
    ...     return None if x == 'world' else x
    ...
    >>> from itemloaders.processors import MapCompose
    >>> proc = MapCompose(filter_world, str.upper)
    >>> proc(['hello', 'world', 'this', 'is', 'something'])
    ['HELLO', 'THIS', 'IS', 'SOMETHING']

    As with the Compose processor, functions can receive Loader contexts, and
    ``__init__`` method keyword arguments are used as default context values.
    See :class:`Compose` processor for more info.

    .. _`parsel selectors`: https://parsel.readthedocs.io/en/latest/parsel.html#parsel.selector.Selector.extract
    """

    def __init__(self, *functions: Callable[..., Any], **default_loader_context: Any):
        self.functions = functions
        self.default_loader_context = default_loader_context

    def __call__(
        self,
        value: Any,
        loader_context: MutableMapping[str, Any] | None = None,
    ) -> Iterable[Any]:
        values = arg_to_iter(value)
        context: MutableMapping[str, Any]
        if loader_context:
            context = ChainMap(loader_context, self.default_loader_context)
        else:
            context = self.default_loader_context
        wrapped_funcs = [wrap_loader_context(f, context) for f in self.functions]
        for func in wrapped_funcs:
            next_values: list[Any] = []
            for v in values:
                try:
                    next_values += arg_to_iter(func(v))
                except Exception as e:
                    raise ValueError(
                        f"Error in MapCompose with "
                        f"{func!s} value={value!r} error='{type(e).__name__}: {e!s}'"
                    ) from e
            values = next_values
        return values

```

--------------------------------

### nested_css()

Source: https://itemloaders.readthedocs.io/en/latest/api-reference.html

Create a nested loader with a css selector. The supplied selector is applied relative to selector associated with this `ItemLoader`. The nested loader shares the item with the parent `ItemLoader` so calls to `add_xpath()`, `add_value()`, `replace_value()`, etc. will behave as expected.

```python
nested_css(_css : str_, _** context: Any_) -> Self[source]
```

--------------------------------

### get_xpath method

Source: https://itemloaders.readthedocs.io/en/latest/_modules/itemloaders.html

Retrieves values using an XPath selector and applies processors.

```python
    def get_xpath(
        self,
        xpath: str | Iterable[str],
        *processors: Callable[..., Any],
        re: str | Pattern[str] | None = None,
        **kw: Any,
    ) -> Any:
        """
        Similar to :meth:`ItemLoader.get_value` but receives an XPath instead of a
        value, which is used to extract a list of unicode strings from the
        selector associated with this :class:`ItemLoader`.

        :param xpath: the XPath to extract data from
        :type xpath: str

        :param re: a regular expression to use for extracting data from the
            selected XPath region
        :type re: str or typing.Pattern[str]

        Examples::

            # HTML snippet: <p class="product-name">Color TV</p>
            loader.get_xpath('//p[@class="product-name"]')
            # HTML snippet: <p id="price">the price is $1200</p>
            loader.get_xpath('//p[@id="price"]', TakeFirst(), re='the price is (.*)')

        """
        values = self._get_xpathvalues(xpath, **kw)
        return self.get_value(values, *processors, re=re, **kw)
```

--------------------------------

### nested_xpath()

Source: https://itemloaders.readthedocs.io/en/latest/api-reference.html

Create a nested loader with an xpath selector. The supplied selector is applied relative to selector associated with this `ItemLoader`. The nested loader shares the item with the parent `ItemLoader` so calls to `add_xpath()`, `add_value()`, `replace_value()`, etc. will behave as expected.

```python
nested_xpath(_xpath : str_, _** context: Any_) -> Self[source]
```

--------------------------------

### get_css method

Source: https://itemloaders.readthedocs.io/en/latest/_modules/itemloaders.html

Retrieves values using a CSS selector and applies processors.

```python
    def get_css(
        self,
        css: str | Iterable[str],
        *processors: Callable[..., Any],
        re: str | Pattern[str] | None = None,
        **kw: Any,
    ) -> Any:
        """
        Similar to :meth:`ItemLoader.get_value` but receives a CSS selector
        instead of a value, which is used to extract a list of unicode strings
        from the selector associated with this :class:`ItemLoader`.

        :param css: the CSS selector to extract data from
        :type css: str

        :param re: a regular expression to use for extracting data from the
            selected CSS region
        :type re: str or typing.Pattern[str]

        Examples::

            # HTML snippet: <p class="product-name">Color TV</p>
            loader.get_css('p.product-name')
            # HTML snippet: <p id="price">the price is $1200</p>
            loader.get_css('p#price', TakeFirst(), re='the price is (.*)')
        """
        values = self._get_cssvalues(css)
        return self.get_value(values, *processors, re=re, **kw)
```

--------------------------------

### Compose Implementation

Source: https://itemloaders.readthedocs.io/en/latest/_modules/itemloaders/processors.html

The implementation of the Compose processor, showing sequential function application.

```python
from __future__ import annotations

from typing import TYPE_CHECKING, Any


if TYPE_CHECKING:
    from collections.abc import Callable, MutableMapping




class Compose:
    """
    A processor which is constructed from the composition of the given
    functions. This means that each input value of this processor is passed to
    the first function, and the result of that function is passed to the second
    function, and so on, until the last function returns the output value of
    this processor.

    By default, stop process on ``None`` value. This behaviour can be changed by
    passing keyword argument ``stop_on_none=False``.

    Example:

    >>> from itemloaders.processors import Compose
    >>> proc = Compose(lambda v: v[0], str.upper)
    >>> proc(['hello', 'world'])
    'HELLO'

    Each function can optionally receive a ``loader_context`` parameter. For
    those which do, this processor will pass the currently active :ref:`Loader
    context <loaders-context>` through that parameter.

    The keyword arguments passed in the ``__init__`` method are used as the default
    Loader context values passed to each function call. However, the final
    Loader context values passed to functions are overridden with the currently
    active Loader context accessible through the :attr:`ItemLoader.context
    <itemloaders.ItemLoader.context>` attribute.
    """

    def __init__(self, *functions: Callable[..., Any], **default_loader_context: Any):
        self.functions = functions
        self.default_loader_context = default_loader_context

    def __call__(self,
                 value: Any,
                 loader_context: MutableMapping[str, Any] | None = None) -> Any:
        context = ChainMap(loader_context or {}, self.default_loader_context)
        for func in self.functions:
            value = func(value, **context)
            if value is None and self.default_loader_context.get('stop_on_none', True):
                break
        return value

```

--------------------------------

### Compose processor

Source: https://itemloaders.readthedocs.io/en/latest/_modules/itemloaders/processors.html

The __init__ and __call__ methods for the Compose processor.

```python
def __init__(self, *functions: Callable[..., Any], **default_loader_context: Any):
        self.functions = functions
        self.stop_on_none = default_loader_context.get("stop_on_none", True)
        self.default_loader_context = default_loader_context

    def __call__(
        self, value: Any, loader_context: MutableMapping[str, Any] | None = None
    ) -> Any:
        context: MutableMapping[str, Any]
        if loader_context:
            context = ChainMap(loader_context, self.default_loader_context)
        else:
            context = self.default_loader_context
        wrapped_funcs = [wrap_loader_context(f, context) for f in self.functions]
        for func in wrapped_funcs:
            if value is None and self.stop_on_none:
                break
            try:
                value = func(value)
            except Exception as e:
                raise ValueError(
                    f"Error in Compose with "
                    f"{func!s} value={value!r} error='{type(e).__name__}: {e!s}'"
                ) from e
        return value
```

--------------------------------

### replace_css()

Source: https://itemloaders.readthedocs.io/en/latest/api-reference.html

Similar to `add_css()` but replaces collected data instead of adding it.

```python
replace_css(_field_name : str | None_, _css : str | Iterable[str]_, _* processors: Callable[..., Any]_, _re : str | Pattern[str] | None = None_, _** kw: Any_) -> Self[source]
```

--------------------------------

### Modifying Item Loader context via attribute

Source: https://itemloaders.readthedocs.io/en/latest/_sources/loaders-context.rst.txt

Shows how to directly modify the Item Loader's context dictionary after instantiation.

```python
loader = ItemLoader(product)
loader.context['unit'] = 'cm'
```

--------------------------------

### ItemLoader load_item method

Source: https://itemloaders.readthedocs.io/en/latest/_modules/itemloaders.html

Method to populate the item with collected data, passing it through output processors.

```python
def load_item(self) -> Any:
    """
    Populate the item with the data collected so far, and return it. The
    data collected is first passed through the :ref:`output processors
    <processors>` to get the final value to assign to each item field.
    """
    adapter = ItemAdapter(self.item)
    for field_name in tuple(self._values):
        value = self.get_output_value(field_name)
        if value is not None:
            adapter[field_name] = value

    return adapter.item
```

--------------------------------

### replace_xpath()

Source: https://itemloaders.readthedocs.io/en/latest/api-reference.html

Similar to `add_xpath()` but replaces collected data instead of adding it.

```python
replace_xpath(_field_name : str | None_, _xpath : str | Iterable[str]_, _* processors: Callable[..., Any]_, _re : str | Pattern[str] | None = None_, _** kw: Any_) -> Self[source]
```

--------------------------------

### replace_jmes()

Source: https://itemloaders.readthedocs.io/en/latest/api-reference.html

Similar to `add_jmes()` but replaces collected data instead of adding it.

```python
replace_jmes(_field_name : str | None_, _jmes : str | Iterable[str]_, _* processors: Callable[..., Any]_, _re : str | Pattern[str] | None = None_, _** kw: Any_) -> Self[source]
```

--------------------------------

### add_value Method

Source: https://itemloaders.readthedocs.io/en/latest/_modules/itemloaders.html

Processes and then adds the given value for the specified field. The value is passed through get_value, then the field input processor, and appended to collected data. Supports adding values for multiple fields if field_name is None.

```python
def add_value(
        self,
        field_name: str | None,
        value: Any,
        *processors: Callable[..., Any],
        re: str | Pattern[str] | None = None,
        **kw: Any,
    ) -> Self:
        """
        Process and then add the given ``value`` for the given field.

        The value is first passed through :meth:`get_value` by giving the
        ``processors`` and ``kwargs``, and then passed through the
        :ref:`field input processor <processors>` and its result
        appended to the data collected for that field. If the field already
        contains collected data, the new data is added.

        The given ``field_name`` can be ``None``, in which case values for
        multiple fields may be added. And the processed value should be a dict
        with field_name mapped to values.

        :returns: The current ItemLoader instance for method chaining.
        :rtype: ItemLoader

        Examples::

            loader.add_value('name', 'Color TV')
            loader.add_value('colours', ['white', 'blue'])
            loader.add_value('length', '100')
            loader.add_value('name', 'name: foo', TakeFirst(), re='name: (.+)')
            loader.add_value(None, {'name': 'foo', 'sex': 'male'})

        """
        value = self.get_value(value, *processors, re=re, **kw)
        if value is None:
            return self
        if not field_name:
            for k, v in value.items():
                self._add_value(k, v)
        else:
            self._add_value(field_name, value)
        return self
```

--------------------------------

### nested_css Method

Source: https://itemloaders.readthedocs.io/en/latest/_modules/itemloaders.html

Creates a nested loader with a css selector, applying it relative to the current selector. The nested loader shares the item with the parent.

```python
def nested_css(self, css: str, **context: Any) -> Self:
    """
    Create a nested loader with a css selector.
    The supplied selector is applied relative to selector associated
    with this :class:`ItemLoader`. The nested loader shares the item
    with the parent :class:`ItemLoader` so calls to :meth:`add_xpath`,
    :meth:`add_value`, :meth:`replace_value`, etc. will behave as expected.
    """
    self._check_selector_method()
    assert self.selector is not None
    selector = self.selector.css(css)
    context.update(selector=selector)
    return self.__class__(item=self.item, parent=self, **context)
```

--------------------------------

### ItemLoader replace_xpath method

Source: https://itemloaders.readthedocs.io/en/latest/_modules/itemloaders.html

Placeholder for the replace_xpath method.

```python
def replace_xpath(
    self,

```

--------------------------------

### get_value Method

Source: https://itemloaders.readthedocs.io/en/latest/_modules/itemloaders.html

Processes the given value by the specified processors and keyword arguments. Supports using a regular expression for data extraction before processing.

```python
def get_value(
        self,
        value: Any,
        *processors: Callable[..., Any],
        re: str | Pattern[str] | None = None,
        **kw: Any,
    ) -> Any:
        """
        Process the given ``value`` by the given ``processors`` and keyword
        arguments.

        Available keyword arguments:

        :param re: a regular expression to use for extracting data from the
            given value using :func:`~parsel.utils.extract_regex` method,
            applied before processors
        :type re: str or typing.Pattern[str]

        Examples:

        >>> from itemloaders import ItemLoader
        >>> from itemloaders.processors import TakeFirst
        
```

--------------------------------

### nested_xpath Method

Source: https://itemloaders.readthedocs.io/en/latest/_modules/itemloaders.html

Creates a nested loader with an xpath selector, applying it relative to the current selector. The nested loader shares the item with the parent.

```python
def nested_xpath(self, xpath: str, **context: Any) -> Self:
    """
    Create a nested loader with an xpath selector.
    The supplied selector is applied relative to selector associated
    with this :class:`ItemLoader`. The nested loader shares the item
    with the parent :class:`ItemLoader` so calls to :meth:`add_xpath`,
    :meth:`add_value`, :meth:`replace_value`, etc. will behave as expected.
    """
    self._check_selector_method()
    assert self.selector is not None
    selector = self.selector.xpath(xpath)
    context.update(selector=selector)
    return self.__class__(item=self.item, parent=self, **context)
```

--------------------------------

### get_jmes Method

Source: https://itemloaders.readthedocs.io/en/latest/_modules/itemloaders.html

Retrieves data using JMESPath extraction.

```python
    def get_jmes(
        self,
        jmes: str | Iterable[str],
        *processors: Callable[..., Any],
        re: str | Pattern[str] | None = None,
        **kw: Any,
    ) -> Any:
        """
        Similar to :meth:`ItemLoader.get_value` but receives a JMESPath selector
        instead of a value, which is used to extract a list of unicode strings
        from the selector associated with this :class:`ItemLoader`.

        :param jmes: the JMESPath selector to extract data from
        :type jmes: str

        :param re: a regular expression to use for extracting data from the
            selected JMESPath
        :type re: str or typing.Pattern

        Examples::

            # HTML snippet: {"name": "Color TV"}
            loader.get_jmes('name')
            # HTML snippet: {"price": the price is $1200"}
            loader.get_jmes('price', TakeFirst(), re='the price is (.*)')
        """
        values = self._get_jmesvalues(jmes)
        return self.get_value(values, *processors, re=re, **kw)

```
Tokens:
Raw