### Install ItemAdapter Source: https://github.com/scrapy/itemadapter/blob/master/README.md Install the itemadapter package using pip. For support with specific libraries like scrapy, attrs, or pydantic, install the corresponding extra. ```bash pip install itemadapter ``` ```bash pip install itemadapter[scrapy] ``` ```bash pip install itemadapter[attrs,pydantic,scrapy] ``` -------------------------------- ### Registering a Custom Adapter Source: https://github.com/scrapy/itemadapter/blob/master/README.md Instructions and examples for registering custom adapter classes with ItemAdapter. ```APIDOC ### Registering an adapter Add your custom adapter class to the `itemadapter.adapter.ItemAdapter.ADAPTER_CLASSES` class attribute in order to handle custom item classes. **Example** ``` pip install zyte-common-items ``` ```python >>> from itemadapter.adapter import ItemAdapter >>> from zyte_common_items import Item, ZyteItemAdapter >>> >>> ItemAdapter.ADAPTER_CLASSES.appendleft(ZyteItemAdapter) >>> item = Item() >>> adapter = ItemAdapter(item) >>> adapter >>> ``` ``` -------------------------------- ### Usage with Dictionaries Source: https://github.com/scrapy/itemadapter/blob/master/README.md Example showing how to use ItemAdapter with Python dictionaries. ```APIDOC ### `dict` ```python >>> from itemadapter import ItemAdapter >>> item = dict(name="foo", price=10) >>> adapter = ItemAdapter(item) >>> adapter.item is item True >>> adapter["name"] 'foo' >>> adapter["name"] = "bar" >>> adapter["price"] = 5 >>> item {'name': 'bar', 'price': 5} >>> ``` ``` -------------------------------- ### Usage with Scrapy Items Source: https://github.com/scrapy/itemadapter/blob/master/README.md Example demonstrating how to use ItemAdapter with Scrapy's Item objects. ```APIDOC ### `scrapy.item.Item` objects ```python >>> from scrapy.item import Item, Field >>> from itemadapter import ItemAdapter >>> class InventoryItem(Item): ... name = Field() ... price = Field() ... >>> item = InventoryItem(name="foo", price=10) >>> adapter = ItemAdapter(item) >>> adapter.item is item True >>> adapter["name"] 'foo' >>> adapter["name"] = "bar" >>> adapter["price"] = 5 >>> item {'name': 'bar', 'price': 5} >>> ``` ``` -------------------------------- ### Working with attrs Classes Source: https://context7.com/scrapy/itemadapter/llms.txt Demonstrates how to use ItemAdapter with attrs-decorated classes, including dict-like operations and accessing field metadata. Ensure attrs is installed. ```python import attr from itemadapter import ItemAdapter @attr.s class Event: name = attr.ib(metadata={"indexed": True}) date = attr.ib(metadata={"format": "ISO8601"}) location = attr.ib(default="TBD", metadata={"optional": True}) attendees = attr.ib(factory=list) # Create and wrap an attrs object event = Event(name="PyCon", date="2024-05-15", location="Pittsburgh") adapter = ItemAdapter(event) # Dict-like operations print(adapter["name"]) adapter["location"] = "Online" adapter["attendees"] = ["Alice", "Bob"] # Access metadata name_meta = adapter.get_field_meta("name") print(name_meta["indexed"]) date_meta = adapter.get_field_meta("date") print(date_meta["format"]) # Field names print(list(adapter.field_names())) # Verify in-place modification print(event) ``` -------------------------------- ### Working with Pydantic Models Source: https://context7.com/scrapy/itemadapter/llms.txt Shows how to use ItemAdapter with Pydantic models (v1 and v2), including dict-like access, field modification, and metadata retrieval. Pydantic must be installed. ```python from pydantic import BaseModel, Field from typing import Optional from itemadapter import ItemAdapter class User(BaseModel): username: str = Field(min_length=3, max_length=20) email: str = Field(pattern=r'^["'\w\.-]+@["'\w\.-]+\."'\w+$') age: Optional[int] = Field(default=None, ge=0, le=150) role: str = Field(default="user", json_schema_extra={"enum": ["user", "admin", "moderator"]}) # Create and wrap a Pydantic model user = User(username="alice", email="alice@example.com", age=28) adapter = ItemAdapter(user) # Dict-like access print(adapter["username"]) print(adapter.get("age")) # Modify fields adapter["role"] = "admin" adapter["age"] = 29 # Access field metadata email_meta = adapter.get_field_meta("email") print("pattern" in email_meta) role_meta = adapter.get_field_meta("role") print(role_meta.get("json_schema_extra")) # Convert to dict print(adapter.asdict()) ``` -------------------------------- ### Register a Custom Adapter Source: https://github.com/scrapy/itemadapter/blob/master/README.md Append your custom adapter class to ItemAdapter.ADAPTER_CLASSES to enable handling of custom item types. Ensure zyte-common-items is installed if using ZyteItemAdapter. ```python pip install zyte-common-items ``` ```python >>> from itemadapter.adapter import ItemAdapter >>> from zyte_common_items import Item, ZyteItemAdapter >>> >>> ItemAdapter.ADAPTER_CLASSES.appendleft(ZyteItemAdapter) >>> item = Item() >>> adapter = ItemAdapter(item) >>> adapter >>> ``` -------------------------------- ### Get All Defined Field Names Source: https://context7.com/scrapy/itemadapter/llms.txt Use `field_names()` to get a `KeysView` of all declared fields for an item. For types with predefined schemas, this includes all fields regardless of their current values. ```python from dataclasses import dataclass, field from typing import Optional from itemadapter import ItemAdapter @dataclass class Article: title: str content: str author: Optional[str] = None tags: list = field(default_factory=list) article = Article(title="Python Tips", content="Some content...") adapter = ItemAdapter(article) # Get all defined field names print(adapter.field_names()) # Output: KeysView({'title': ..., 'content': ..., 'author': ..., 'tags': ...}) print(list(adapter.field_names())) # Output: ['title', 'content', 'author', 'tags'] # Compare with iteration (only populated fields for some types) print(list(adapter.keys())) # Output: ['title', 'content', 'author', 'tags'] ``` -------------------------------- ### Get Field Names from Class Source: https://context7.com/scrapy/itemadapter/llms.txt Use the class method `get_field_names_from_class()` to get a list of all field names defined for an item class. Returns `None` for types without predefined fields, like plain dictionaries. ```python from dataclasses import dataclass from itemadapter import ItemAdapter @dataclass class Employee: id: int name: str department: str salary: float # Get field names from class fields = ItemAdapter.get_field_names_from_class(Employee) print(fields) # Output: ['id', 'name', 'department', 'salary'] # Dict class returns None (no predefined fields) dict_fields = ItemAdapter.get_field_names_from_class(dict) print(dict_fields) # Output: None # Useful for generating forms, schemas, or validation rules for field_name in ItemAdapter.get_field_names_from_class(Employee) or []: print(f"Field: {field_name}") # Output: # Field: id # Field: name # Field: department # Field: salary ``` -------------------------------- ### Check Item Type with ThirdPartyItemAdapter Source: https://context7.com/scrapy/itemadapter/llms.txt Use ThirdPartyItemAdapter.is_item() to check if an object is a supported item type by third-party adapters. This example shows it does not support dataclasses or dicts. ```python from itemadapter.adapter import ThirdPartyItemAdapter from dataclasses import dataclass @dataclass class DataclassItem: name: str dc_item = DataclassItem(name="test") dict_item = {"name": "dict test"} print(ThirdPartyItemAdapter.is_item(dc_item)) # Output: False print(ThirdPartyItemAdapter.is_item(dict_item)) # Output: False ``` -------------------------------- ### Get Field Metadata Source: https://context7.com/scrapy/itemadapter/llms.txt Use `get_field_meta()` to retrieve metadata for a specific field. Metadata is returned as a read-only `MappingProxyType`. ```python from dataclasses import dataclass, field from itemadapter import ItemAdapter @dataclass class Product: name: str = field(metadata={"serializer": str, "max_length": 100}) price: float = field(metadata={"serializer": float, "min_value": 0}) sku: str = field(default="", metadata={"unique": True, "indexed": True}) product = Product(name="Widget", price=19.99, sku="WGT-001") adapter = ItemAdapter(product) # Access field metadata name_meta = adapter.get_field_meta("name") print(name_meta) # Output: mappingproxy({'serializer': , 'max_length': 100}) price_meta = adapter.get_field_meta("price") print(price_meta["min_value"]) # Output: 0 sku_meta = adapter.get_field_meta("sku") print(sku_meta["unique"]) # Output: True print(sku_meta["indexed"]) # Output: True # Metadata is read-only try: name_meta["max_length"] = 200 except TypeError as e: print(f"Cannot modify: {e}") # Output: Cannot modify: 'mappingproxy' object does not support item assignment ``` -------------------------------- ### Get Field Metadata from Class Source: https://context7.com/scrapy/itemadapter/llms.txt Use the class method `get_field_meta_from_class()` to retrieve field metadata directly from an item class without an instance. This is useful for schema inspection. ```python from dataclasses import dataclass, field from itemadapter import ItemAdapter @dataclass class Order: order_id: str = field(metadata={"primary_key": True}) items: list = field(default_factory=list, metadata={"element_type": "OrderItem"}) total: float = field(default=0.0, metadata={"currency": "USD", "precision": 2}) # Get metadata without creating an instance order_id_meta = ItemAdapter.get_field_meta_from_class(Order, "order_id") print(order_id_meta) # Output: mappingproxy({'primary_key': True}) total_meta = ItemAdapter.get_field_meta_from_class(Order, "total") print(f"Currency: {total_meta['currency']}, Precision: {total_meta['precision']}") # Output: Currency: USD, Precision: 2 # Introspect all fields field_names = ItemAdapter.get_field_names_from_class(Order) for name in field_names: meta = ItemAdapter.get_field_meta_from_class(Order, name) print(f"{name}: {dict(meta)}") # Output: # order_id: {'primary_key': True} # items: {'element_type': 'OrderItem'} # total: {'currency': 'USD', 'precision': 2} ``` -------------------------------- ### Subclassing ItemAdapter for Specialized Configurations Source: https://context7.com/scrapy/itemadapter/llms.txt Demonstrates creating specialized ItemAdapter subclasses by defining custom ADAPTER_CLASSES lists. This allows for tailored adapter behavior based on specific item types. ```python from itemadapter.adapter import ( ItemAdapter, DictAdapter, DataclassAdapter, AttrsAdapter, PydanticAdapter, ScrapyItemAdapter, ) # Adapter for built-in Python types only class BuiltinItemAdapter(ItemAdapter): ADAPTER_CLASSES = [DictAdapter, DataclassAdapter] # Adapter for third-party library types only class ThirdPartyItemAdapter(ItemAdapter): ADAPTER_CLASSES = [AttrsAdapter, PydanticAdapter, ScrapyItemAdapter] from dataclasses import dataclass @dataclass class DataclassItem: name: str ``` -------------------------------- ### ItemAdapter Class Usage Source: https://context7.com/scrapy/itemadapter/llms.txt Demonstrates how to use the ItemAdapter class to wrap data objects and access their fields using a dict-like interface. ```APIDOC ## ItemAdapter Class The main entry point for wrapping data container objects. Creates an adapter that provides dict-like access to the wrapped item's fields, automatically detecting the appropriate underlying adapter based on the item's type. ### Method `ItemAdapter(item)` ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example ```python from dataclasses import dataclass from itemadapter import ItemAdapter @dataclass class Product: name: str price: float stock: int # Create an item and wrap it item = Product(name="Widget", price=29.99, stock=100) adapter = ItemAdapter(item) # Dict-like read access print(adapter["name"]) print(adapter.get("price")) print(len(adapter)) # Dict-like write access (modifies original item in-place) adapter["price"] = 24.99 adapter["stock"] = 85 adapter.update({"name": "Super Widget"}) print(item) print(adapter.item is item) # Iteration over fields for field_name in adapter: print(f"{field_name}: {adapter[field_name]}") # Convert to dictionary print(dict(adapter)) ``` ### Response #### Success Response (200) ItemAdapter instance providing dict-like access to the item. #### Response Example ```json { "example": "ItemAdapter instance" } ``` ``` -------------------------------- ### Customizing Adapter Priorities Source: https://github.com/scrapy/itemadapter/blob/master/README.md Demonstrates how to subclass ItemAdapter to define custom adapter class lists and priorities. ```APIDOC ### Multiple adapter classes If you need to have different handlers and/or priorities for different cases you can subclass the `ItemAdapter` class and set the `ADAPTER_CLASSES` attribute as needed: **Example** ```python >>> from itemadapter.adapter import ( ... ItemAdapter, ... AttrsAdapter, ... DataclassAdapter, ... DictAdapter, ... PydanticAdapter, ... ScrapyItemAdapter, ... ) >>> from scrapy.item import Item, Field >>> >>> class BuiltinTypesItemAdapter(ItemAdapter): ... ADAPTER_CLASSES = [DictAdapter, DataclassAdapter] ... >>> class ThirdPartyTypesItemAdapter(ItemAdapter): ... ADAPTER_CLASSES = [AttrsAdapter, PydanticAdapter, ScrapyItemAdapter] ... >>> class ScrapyItem(Item): ... foo = Field() ... >>> BuiltinTypesItemAdapter.is_item(dict()) True >>> ThirdPartyTypesItemAdapter.is_item(dict()) False >>> BuiltinTypesItemAdapter.is_item(ScrapyItem(foo="bar")) False >>> ThirdPartyTypesItemAdapter.is_item(ScrapyItem(foo="bar")) True >>> ``` ``` -------------------------------- ### Basic ItemAdapter Usage with Dataclass Source: https://github.com/scrapy/itemadapter/blob/master/README.md Demonstrates initializing an ItemAdapter with a dataclass object and accessing its properties like a dictionary. The wrapped object is modified in-place. ```python from dataclasses import dataclass from itemadapter import ItemAdapter @dataclass class InventoryItem: name: str price: float stock: int obj = InventoryItem(name='foo', price=20.5, stock=10) ItemAdapter.is_item(obj) adapter = ItemAdapter(obj) len(adapter) adapter["name"] adapter.get("price") ``` ```python adapter["name"] = "bar" adapter.update({"price": 12.7, "stock": 9}) adapter.item is obj ``` -------------------------------- ### Custom Adapter Registration Source: https://context7.com/scrapy/itemadapter/llms.txt Illustrates how to extend ItemAdapter for custom item types by implementing AdapterInterface and registering it. This allows ItemAdapter to work with user-defined classes. ```python from collections.abc import Iterator, KeysView from types import MappingProxyType from itemadapter.adapter import ItemAdapter, AdapterInterface # Custom item class class CustomRecord: def __init__(self, **kwargs): self._data = kwargs def __repr__(self): return f"CustomRecord({self._data})" # Custom adapter implementation class CustomRecordAdapter(AdapterInterface): @classmethod def is_item_class(cls, item_class: type) -> bool: return item_class is CustomRecord @classmethod def is_item(cls, item) -> bool: return isinstance(item, CustomRecord) @classmethod def get_field_names_from_class(cls, item_class: type) -> list[str] | None: return None # Dynamic fields, no predefined schema def __getitem__(self, field_name: str): return self.item._data[field_name] def __setitem__(self, field_name: str, value): self.item._data[field_name] = value def __delitem__(self, field_name: str): del self.item._data[field_name] def __iter__(self) -> Iterator: return iter(self.item._data) def __len__(self) -> int: return len(self.item._data) def field_names(self) -> KeysView: return KeysView(self.item._data) # Register the custom adapter (prepend for priority) ItemAdapter.ADAPTER_CLASSES.appendleft(CustomRecordAdapter) # Now ItemAdapter works with CustomRecord record = CustomRecord(id=1, name="Test", value=42) adapter = ItemAdapter(record) print(ItemAdapter.is_item(record)) print(adapter["name"]) adapter["status"] = "active" print(adapter.asdict()) ``` -------------------------------- ### ItemAdapter Methods Source: https://github.com/scrapy/itemadapter/blob/master/README.md Core methods for interacting with item instances via the ItemAdapter. ```APIDOC ## ItemAdapter Methods ### get_field_meta(field_name: str) -> MappingProxyType Returns metadata for the specified field. ### field_names() -> collections.abc.KeysView Returns a keys view containing the names of all defined fields for the item. ### asdict() -> dict Returns a dictionary representation of the adapter contents, applied recursively to nested items. ``` -------------------------------- ### Adapter Interface Source: https://github.com/scrapy/itemadapter/blob/master/README.md Defines the abstract base class for adapters and its methods. ```APIDOC ## class itemadapter.adapter.AdapterInterface(item: Any) ### Description Abstract Base Class for adapters. An adapter that handles a specific type of item must inherit from this class and implement the abstract methods defined on it. `AdapterInterface` inherits from [`collections.abc.MutableMapping`](https://docs.python.org/3/library/collections.abc.html#collections.abc.MutableMapping), so all methods from the `MutableMapping` interface must be implemented as well. ### Methods * _class method `is_item_class(cls, item_class: type) -> bool`_ Return `True` if the adapter can handle the given item class, `False` otherwise. Abstract (mandatory). * _class method `is_item(cls, item: Any) -> bool`_ Return `True` if the adapter can handle the given item, `False` otherwise. The default implementation calls `cls.is_item_class(item.__class__)`. * _class method `get_field_meta_from_class(cls, item_class: type) -> bool`_ Return metadata for the given item class and field name, if available. By default, this method returns an empty `MappingProxyType` object. Please supply your own method definition if you want to handle field metadata based on custom logic. See the [section on metadata support](#metadata-support) for additional information. * _method `get_field_meta(self, field_name: str) -> types.MappingProxyType`_ Return metadata for the given field name, if available. It's usually not necessary to override this method, since the `itemadapter.adapter.AdapterInterface` base class provides a default implementation that calls `ItemAdapter.get_field_meta_from_class` with the wrapped item's class as argument. See the [section on metadata support](#metadata-support) for additional information. * _method `field_names(self) -> collections.abc.KeysView`_: Return a [dynamic view](https://docs.python.org/3/library/collections.abc.html#collections.abc.KeysView) of the item's field names. By default, this method returns the result of calling `keys()` on the current adapter, i.e., its return value depends on the implementation of the methods from the `MutableMapping` interface (more specifically, it depends on the return value of `__iter__`). You might want to override this method if you want a way to get all fields for an item, whether or not they are populated. For instance, Scrapy uses this method to define column names when exporting items to CSV. ``` -------------------------------- ### Using ItemAdapter with dataclass objects Source: https://github.com/scrapy/itemadapter/blob/master/README.md Demonstrates initializing ItemAdapter with a dataclass object and accessing/modifying its fields. ```python from dataclasses import dataclass from itemadapter import ItemAdapter @dataclass class InventoryItem: name: str price: int item = InventoryItem(name="foo", price=10) adapter = ItemAdapter(item) print(adapter.item is item) print(adapter["name"]) adapter["name"] = "bar" adapter["price"] = 5 print(item) ``` -------------------------------- ### ItemAdapter with Scrapy Items Source: https://context7.com/scrapy/itemadapter/llms.txt Demonstrates how ItemAdapter seamlessly works with Scrapy items, providing the same interface and accessing field metadata from Scrapy's Field definitions. ```APIDOC ## Working with Scrapy Items ### Description ItemAdapter seamlessly works with Scrapy items, providing the same interface as for other item types. Field metadata comes from Scrapy's Field definitions. ### Method `ItemAdapter(scrapy_item)` ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example ```python from scrapy.item import Item, Field from itemadapter import ItemAdapter class ProductItem(Item): name = Field(serializer=str) price = Field(serializer=float, output_processor=lambda x: round(x, 2)) url = Field() category = Field(default="uncategorized") # Create and wrap a Scrapy item item = ProductItem(name="Laptop", price=999.99, url="https://example.com/laptop") adapter = ItemAdapter(item) # Dict-like access print(adapter["name"]) print(adapter.get("url")) # Modify fields adapter["price"] = 899.99 adapter["category"] = "electronics" # Access field metadata price_meta = adapter.get_field_meta("price") print(price_meta["serializer"]) # Get all field names print(list(adapter.field_names())) # Convert to dict for export print(adapter.asdict()) ``` ### Response #### Success Response (200) - **adapter** (ItemAdapter) - An ItemAdapter instance wrapping the Scrapy item. #### Response Example ``` Laptop https://example.com/laptop ['name', 'price', 'url', 'category'] {'name': 'Laptop', 'price': 899.99, 'url': 'https://example.com/laptop', 'category': 'electronics'} ``` ``` -------------------------------- ### Using ItemAdapter with attrs objects Source: https://github.com/scrapy/itemadapter/blob/master/README.md Shows how to use ItemAdapter with objects defined using the attrs library, including field access and modification. ```python import attr from itemadapter import ItemAdapter @attr.s class InventoryItem: name = attr.ib() price = attr.ib() item = InventoryItem(name="foo", price=10) adapter = ItemAdapter(item) print(adapter.item is item) print(adapter["name"]) adapter["name"] = "bar" adapter["price"] = 5 print(item) ``` -------------------------------- ### Utility Functions Source: https://github.com/scrapy/itemadapter/blob/master/README.md Helper functions for checking item types and retrieving metadata from classes. ```APIDOC ## Utility Functions ### is_item(obj: Any) -> bool Returns True if the object belongs to a supported item type. ### get_field_meta_from_class(item_class: type, field_name: str) -> types.MappingProxyType Retrieves field metadata directly from the item class. ``` -------------------------------- ### Access Item Field using BuiltinItemAdapter Source: https://context7.com/scrapy/itemadapter/llms.txt Instantiate BuiltinItemAdapter with an item and access its fields using dictionary-like syntax. This adapter handles dataclasses and dicts. ```python from itemadapter.adapter import BuiltinItemAdapter from dataclasses import dataclass @dataclass class DataclassItem: name: str dc_item = DataclassItem(name="test") builtin_adapter = BuiltinItemAdapter(dc_item) print(builtin_adapter["name"]) # Output: test ``` -------------------------------- ### Generate JSON Schema for an Item Class Source: https://github.com/scrapy/itemadapter/blob/master/README.md Demonstrates how to generate a JSON schema representation for dataclass and attrs-based item classes. ```python from dataclasses import dataclass import attrs @dataclass class Brand: name: str @attrs.define class Product: name: str """Product name""" brand: Brand | None in_stock: bool = True ``` ```python { "type": "object", "additionalProperties": False, "properties": { "name": {"type": "string", "description": "Product name"}, "brand": { "anyOf": [ {"type": "null"}, { "type": "object", "additionalProperties": False, "properties": {"name": {"type": "string"}}, "required": ["name"], }, ] }, "in_stock": {"default": True, "type": "boolean"}, }, "required": ["name", "brand"], } ``` -------------------------------- ### ItemAdapter.asdict() Method Source: https://context7.com/scrapy/itemadapter/llms.txt Converts the wrapped item to a dictionary, recursively converting any nested items. ```APIDOC ## ItemAdapter.asdict() Converts the wrapped item to a dictionary, recursively converting any nested items. Unlike `dict(adapter)`, this method properly handles nested item objects. ### Method `adapter.asdict()` ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example ```python from dataclasses import dataclass from itemadapter import ItemAdapter @dataclass class Address: street: str city: str country: str @dataclass class Customer: name: str email: str billing_address: Address shipping_address: Address # Create nested item structure customer = Customer( name="Alice Smith", email="alice@example.com", billing_address=Address("123 Main St", "New York", "USA"), shipping_address=Address("456 Oak Ave", "Boston", "USA"), ) adapter = ItemAdapter(customer) # Convert to dictionary with nested conversion dict_representation = adapter.asdict() print(dict_representation) ``` ### Response #### Success Response (200) A dictionary representation of the item, including nested items. #### Response Example ```json { "example": "{\"name\": \"Alice Smith\", \"email\": \"alice@example.com\", \"billing_address\": {\"street\": \"123 Main St\", \"city\": \"New York\", \"country\": \"USA\"}, \"shipping_address\": {\"street\": \"456 Oak Ave\", \"city\": \"Boston\", \"country\": \"USA\"}}" } ``` ``` -------------------------------- ### ItemAdapter Class Methods Source: https://github.com/scrapy/itemadapter/blob/master/README.md Details on the class methods available in the ItemAdapter class for handling items and retrieving field information. ```APIDOC ## ItemAdapter Class Methods ### Description This section describes the class methods of the `ItemAdapter` class, which are used for interacting with registered adapters and retrieving item metadata. ### ADAPTER_CLASSES #### class attribute `ADAPTER_CLASSES: Iterable` Stores the currently registered adapter classes. The order is important for determining which adapter handles an item first. It defaults to a `collections.deque` but can be any iterable for subclasses. ### is_item #### class method `is_item(item: Any) -> bool` **Description**: Returns `True` if any of the registered adapters can handle the given item, `False` otherwise. ### is_item_class #### class method `is_item_class(item_class: type) -> bool` **Description**: Returns `True` if any of the registered adapters can handle the given item class, `False` otherwise. ### get_field_meta_from_class #### class method `get_field_meta_from_class(item_class: type, field_name: str) -> MappingProxyType` **Description**: Returns a read-only mapping (`MappingProxyType`) containing metadata for the specified field within the given item class. It checks various sources like `scrapy.item.Field`, `dataclasses.field.metadata`, `attr.Attribute.metadata`, and `pydantic.fields.FieldInfo`. Returns an empty object if the item class does not support field metadata or if the field has no metadata. ### get_field_names_from_class #### class method `get_field_names_from_class(item_class: type) -> Optional[list[str]]` **Description**: Returns a list of field names defined for the item class. Returns `None` if the item class does not support defining fields upfront. ``` -------------------------------- ### Interact with Scrapy Items Source: https://context7.com/scrapy/itemadapter/llms.txt Demonstrates using ItemAdapter to manipulate Scrapy items, access field metadata, and convert items to dictionaries. ```python from scrapy.item import Item, Field from itemadapter import ItemAdapter class ProductItem(Item): name = Field(serializer=str) price = Field(serializer=float, output_processor=lambda x: round(x, 2)) url = Field() category = Field(default="uncategorized") # Create and wrap a Scrapy item item = ProductItem(name="Laptop", price=999.99, url="https://example.com/laptop") adapter = ItemAdapter(item) # Dict-like access print(adapter["name"]) # Output: Laptop print(adapter.get("url")) # Output: https://example.com/laptop # Modify fields adapter["price"] = 899.99 adapter["category"] = "electronics" # Access field metadata price_meta = adapter.get_field_meta("price") print(price_meta["serializer"]) # Output: # Get all field names print(list(adapter.field_names())) # Output: ['name', 'price', 'url', 'category'] # Convert to dict for export print(adapter.asdict()) # Output: {'name': 'Laptop', 'price': 899.99, 'url': 'https://example.com/laptop', 'category': 'electronics'} ``` -------------------------------- ### Using ItemAdapter with Pydantic objects Source: https://github.com/scrapy/itemadapter/blob/master/README.md Illustrates the usage of ItemAdapter with Pydantic BaseModel objects for field manipulation. ```python from pydantic import BaseModel from itemadapter import ItemAdapter class InventoryItem(BaseModel): name: str price: int item = InventoryItem(name="foo", price=10) adapter = ItemAdapter(item) print(adapter.item is item) print(adapter["name"]) adapter["name"] = "bar" adapter["price"] = 5 print(item) ``` -------------------------------- ### Check Item Type with BuiltinItemAdapter Source: https://context7.com/scrapy/itemadapter/llms.txt Use BuiltinItemAdapter.is_item() to check if an object is a supported item type (e.g., dataclass or dict). This is useful for filtering items before processing. ```python from itemadapter.adapter import BuiltinItemAdapter from dataclasses import dataclass @dataclass class DataclassItem: name: str dc_item = DataclassItem(name="test") dict_item = {"name": "dict test"} print(BuiltinItemAdapter.is_item(dc_item)) # Output: True print(BuiltinItemAdapter.is_item(dict_item)) # Output: True ``` -------------------------------- ### Validate item classes with ItemAdapter.is_item_class() Source: https://context7.com/scrapy/itemadapter/llms.txt Verifies if a class definition is compatible with ItemAdapter. This is useful for pre-instantiation validation. ```python from dataclasses import dataclass from itemadapter import ItemAdapter @dataclass class Product: name: str price: float class RegularClass: def __init__(self, value): self.value = value # Check if classes can produce valid items print(ItemAdapter.is_item_class(dict)) # Output: True print(ItemAdapter.is_item_class(Product)) # Output: True print(ItemAdapter.is_item_class(RegularClass)) # Output: False print(ItemAdapter.is_item_class(str)) # Output: False print(ItemAdapter.is_item_class(list)) # Output: False ``` -------------------------------- ### ItemAdapter.field_names() Source: https://context7.com/scrapy/itemadapter/llms.txt Returns a KeysView containing all field names defined for the item, including those without values for schema-based types. ```APIDOC ## ItemAdapter.field_names() ### Description Returns a `KeysView` containing all field names defined for the item. For types with predefined schemas (dataclass, attrs, pydantic, Scrapy), this includes all declared fields regardless of whether they have values. ### Method Method call on ItemAdapter instance ### Response - **Returns** (KeysView) - A collection of all field names defined in the item schema. ``` -------------------------------- ### Extend JSON Schema with Field Metadata Source: https://github.com/scrapy/itemadapter/blob/master/README.md Shows how to use json_schema_extra in Field metadata to customize schema generation for specific fields. ```python >>> from scrapy.item import Item, Field >>> from itemadapter import ItemAdapter >>> class MyItem(Item): ... name: str = Field(json_schema_extra={"minLength": 1}) ... >>> ItemAdapter.get_json_schema(MyItem) {'type': 'object', 'additionalProperties': False, 'properties': {'name': {'minLength': 1, 'type': 'string'}}} ``` -------------------------------- ### Wrap and manipulate data with ItemAdapter Source: https://context7.com/scrapy/itemadapter/llms.txt Wraps an object to provide a dict-like interface for reading, writing, and iterating over fields. Modifications to the adapter are applied to the original object in-place. ```python from dataclasses import dataclass from itemadapter import ItemAdapter @dataclass class Product: name: str price: float stock: int # Create an item and wrap it item = Product(name="Widget", price=29.99, stock=100) adapter = ItemAdapter(item) # Dict-like read access print(adapter["name"]) # Output: Widget print(adapter.get("price")) # Output: 29.99 print(len(adapter)) # Output: 3 # Dict-like write access (modifies original item in-place) adapter["price"] = 24.99 adapter["stock"] = 85 adapter.update({"name": "Super Widget"}) print(item) # Output: Product(name='Super Widget', price=24.99, stock=85) print(adapter.item is item) # Output: True # Iteration over fields for field_name in adapter: print(f"{field_name}: {adapter[field_name]}") # Output: # name: Super Widget # price: 24.99 # stock: 85 # Convert to dictionary print(dict(adapter)) # Output: {'name': 'Super Widget', 'price': 24.99, 'stock': 85} ``` -------------------------------- ### Using ItemAdapter with Scrapy Items Source: https://github.com/scrapy/itemadapter/blob/master/README.md Instantiate ItemAdapter with a Scrapy Item object to access and modify its fields. The adapter provides dictionary-like access to item fields. ```python >>> from scrapy.item import Item, Field >>> from itemadapter import ItemAdapter >>> class InventoryItem(Item): ... name = Field() ... price = Field() ... >>> item = InventoryItem(name="foo", price=10) >>> adapter = ItemAdapter(item) >>> adapter.item is item True >>> adapter["name"] 'foo' >>> adapter["name"] = "bar" >>> adapter["price"] = 5 >>> item {'name': 'bar', 'price': 5} >>> ``` -------------------------------- ### ItemAdapter.is_item_class() Class Method Source: https://context7.com/scrapy/itemadapter/llms.txt Checks if a given class can be used to create items that ItemAdapter can handle. ```APIDOC ## ItemAdapter.is_item_class() Class method that checks if a given class can be used to create items that ItemAdapter can handle. Useful for validating item class definitions before instantiation. ### Method `ItemAdapter.is_item_class(cls)` ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example ```python from dataclasses import dataclass from itemadapter import ItemAdapter @dataclass class Product: name: str price: float class RegularClass: def __init__(self, value): self.value = value # Check if classes can produce valid items print(ItemAdapter.is_item_class(dict)) print(ItemAdapter.is_item_class(Product)) print(ItemAdapter.is_item_class(RegularClass)) print(ItemAdapter.is_item_class(str)) print(ItemAdapter.is_item_class(list)) ``` ### Response #### Success Response (200) `True` if the class can produce supported items, `False` otherwise. #### Response Example ```json { "example": "True or False" } ``` ``` -------------------------------- ### Define Custom ItemAdapter Subclasses Source: https://github.com/scrapy/itemadapter/blob/master/README.md Subclass ItemAdapter to define custom adapter priorities and handlers for different item types. Use ADAPTER_CLASSES to specify the order of adapters to be checked. ```python >>> from itemadapter.adapter import ( ... ItemAdapter, ... AttrsAdapter, ... DataclassAdapter, ... DictAdapter, ... PydanticAdapter, ... ScrapyItemAdapter, ... ) >>> from scrapy.item import Item, Field >>> >>> class BuiltinTypesItemAdapter(ItemAdapter): ... ADAPTER_CLASSES = [DictAdapter, DataclassAdapter] ... >>> class ThirdPartyTypesItemAdapter(ItemAdapter): ... ADAPTER_CLASSES = [AttrsAdapter, PydanticAdapter, ScrapyItemAdapter] ... >>> class ScrapyItem(Item): ... foo = Field() ... >>> BuiltinTypesItemAdapter.is_item(dict()) True >>> ThirdPartyTypesItemAdapter.is_item(dict()) False >>> BuiltinTypesItemAdapter.is_item(ScrapyItem(foo="bar")) False >>> ThirdPartyTypesItemAdapter.is_item(ScrapyItem(foo="bar")) True >>> ``` -------------------------------- ### Convert nested items with ItemAdapter.asdict() Source: https://context7.com/scrapy/itemadapter/llms.txt Converts an item and its nested structures into a dictionary. This method handles recursive conversion of nested item objects. ```python from dataclasses import dataclass from itemadapter import ItemAdapter @dataclass class Address: street: str city: str country: str @dataclass class Customer: name: str email: str billing_address: Address shipping_address: Address # Create nested item structure customer = Customer( name="Alice Smith", email="alice@example.com", billing_address=Address("123 Main St", "New York", "USA"), shipping_address=Address("456 Oak Ave", "Boston", "USA"), ) adapter = ItemAdapter(customer) ``` -------------------------------- ### ItemAdapter.get_json_schema() Source: https://context7.com/scrapy/itemadapter/llms.txt Generates a JSON Schema representation of an item class, including type information, required fields, defaults, descriptions from docstrings, and validation constraints. ```APIDOC ## ItemAdapter.get_json_schema() ### Description Class method that generates a JSON Schema representation of an item class. Includes type information, required fields, defaults, descriptions from docstrings, and validation constraints. ### Method `ItemAdapter.get_json_schema(item_class) ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example ```python from dataclasses import dataclass import json from itemadapter import ItemAdapter @dataclass class Address: street: str city: str postal_code: str @dataclass class Person: name: str """The person's full name""" age: int email: str | None = None address: Address | None = None active: bool = True # Generate JSON Schema schema = ItemAdapter.get_json_schema(Person) print(json.dumps(schema, indent=2)) ``` ### Response #### Success Response (200) - **schema** (object) - A JSON Schema object representing the item class. #### Response Example ```json { "type": "object", "additionalProperties": false, "properties": { "name": { "type": "string", "description": "The person's full name" }, "age": { "type": "integer" }, "email": { "type": ["null", "string"] }, "address": { "anyOf": [ {"type": "null"}, { "type": "object", "additionalProperties": false, "properties": { "street": {"type": "string"}, "city": {"type": "string"}, "postal_code": {"type": "string"} }, "required": ["street", "city", "postal_code"] } ] }, "active": { "default": true, "type": "boolean" } }, "required": ["name", "age"] } ``` ``` -------------------------------- ### ItemAdapter.is_item() Class Method Source: https://context7.com/scrapy/itemadapter/llms.txt Checks if a given object is a supported item type that ItemAdapter can handle. ```APIDOC ## ItemAdapter.is_item() Class method that checks if a given object can be handled by any of the registered adapters. Returns `True` if the object is a supported item type (dict, dataclass, attrs, pydantic, or Scrapy item), `False` otherwise. ### Method `ItemAdapter.is_item(obj)` ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example ```python from dataclasses import dataclass from itemadapter import ItemAdapter @dataclass class InventoryItem: name: str quantity: int # Check various object types print(ItemAdapter.is_item({"key": "value"})) print(ItemAdapter.is_item(InventoryItem("Bolt", 500))) print(ItemAdapter.is_item("just a string")) print(ItemAdapter.is_item(42)) print(ItemAdapter.is_item([1, 2, 3])) # Useful for filtering items in a processing pipeline items = [ {"name": "Widget", "price": 10}, "not an item", InventoryItem("Gadget", 25), None, ] valid_items = [item for item in items if ItemAdapter.is_item(item)] for item in valid_items: adapter = ItemAdapter(item) print(f"Processing: {adapter.get('name')}") ``` ### Response #### Success Response (200) `True` if the object is a supported item type, `False` otherwise. #### Response Example ```json { "example": "True or False" } ``` ``` -------------------------------- ### Convert ItemAdapter to Dictionary Recursively Source: https://github.com/scrapy/itemadapter/blob/master/README.md Use the asdict method to convert an ItemAdapter and its nested items into a dictionary. Note that the dict() built-in does not perform recursive conversion. ```python from dataclasses import dataclass from itemadapter import ItemAdapter @dataclass class Price: value: int currency: str @dataclass class Product: name: str price: Price item = Product("Stuff", Price(42, "UYU")) adapter = ItemAdapter(item) adapter.asdict() ``` ```python dict(adapter) ``` -------------------------------- ### ItemAdapter.get_json_schema Source: https://github.com/scrapy/itemadapter/blob/master/README.md Generates a JSON Schema representation of an item class, reflecting type hints, docstrings, and metadata. ```APIDOC ## get_json_schema(item_class: type) -> dict[str, Any] ### Description Returns a dictionary containing the JSON Schema representation of the provided item class. ### Parameters #### Request Body - **item_class** (type) - Required - The class to generate the schema for. ### Response #### Success Response (200) - **schema** (dict) - A dictionary representing the JSON Schema. ``` -------------------------------- ### ItemAdapter.get_field_names_from_class() Source: https://context7.com/scrapy/itemadapter/llms.txt Class method that returns a list of all field names defined for an item class. ```APIDOC ## ItemAdapter.get_field_names_from_class(item_class) ### Description Class method that returns a list of all field names defined for an item class. Returns `None` for types that don't support predefined field definitions. ### Parameters #### Path Parameters - **item_class** (type) - Required - The class definition to inspect. ### Response - **Returns** (list|None) - A list of field names or None if the type does not support predefined fields. ``` -------------------------------- ### Using ItemAdapter with Dictionaries Source: https://github.com/scrapy/itemadapter/blob/master/README.md ItemAdapter can directly handle Python dictionaries, providing a consistent interface for accessing and modifying key-value pairs. ```python >>> from itemadapter import ItemAdapter >>> item = dict(name="foo", price=10) >>> adapter = ItemAdapter(item) >>> adapter.item is item True >>> adapter["name"] 'foo' >>> adapter["name"] = "bar" >>> adapter["price"] = 5 >>> item {'name': 'bar', 'price': 5} >>> ``` -------------------------------- ### Generate JSON Schema from Dataclass Source: https://context7.com/scrapy/itemadapter/llms.txt Uses ItemAdapter.get_json_schema to create a JSON schema representation of a dataclass, including field types and docstring descriptions. ```python from dataclasses import dataclass import json from itemadapter import ItemAdapter @dataclass class Address: street: str city: str postal_code: str @dataclass class Person: name: str """The person's full name""" age: int email: str | None = None address: Address | None = None active: bool = True # Generate JSON Schema schema = ItemAdapter.get_json_schema(Person) print(json.dumps(schema, indent=2)) ``` -------------------------------- ### ItemAdapter.get_field_meta_from_class() Source: https://context7.com/scrapy/itemadapter/llms.txt Class method to retrieve field metadata directly from an item class without needing an instance. ```APIDOC ## ItemAdapter.get_field_meta_from_class(item_class, field_name) ### Description Class method to retrieve field metadata directly from an item class without needing an instance. Useful for schema inspection and validation logic. ### Parameters #### Path Parameters - **item_class** (type) - Required - The class definition to inspect. - **field_name** (str) - Required - The name of the field to retrieve metadata for. ### Response - **Returns** (MappingProxyType) - A read-only dictionary-like object containing the field's metadata. ``` -------------------------------- ### is_item() Utility Function Source: https://context7.com/scrapy/itemadapter/llms.txt Standalone utility function that checks if an object is a valid item. This is an alias for `ItemAdapter.is_item()` provided for convenience. ```APIDOC ## is_item() Utility Function ### Description Standalone utility function that checks if an object is a valid item. This is an alias for `ItemAdapter.is_item()` provided for convenience. ### Method `is_item(obj) ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example ```python from dataclasses import dataclass from itemadapter import is_item @dataclass class Book: title: str author: str # Check various types print(is_item({"key": "value"})) # Output: True print(is_item(Book("1984", "Orwell"))) # Output: True print(is_item("string")) # Output: False print(is_item(123)) # Output: False # Filter valid items from mixed data def process_items(data_list): for data in data_list: if is_item(data): yield data mixed_data = [ {"title": "Dict Item"}, Book("Dune", "Herbert"), "invalid", None, 42, ] for item in process_items(mixed_data): print(f"Valid item: {item}") ``` ### Response #### Success Response (200) - **is_item** (boolean) - True if the object is a valid item, False otherwise. #### Response Example ``` True False Valid item: {'title': 'Dict Item'} Valid item: Book(title='Dune', author='Herbert') ``` ```