### Install feedparser using pip Source: https://github.com/kurtmckee/feedparser/blob/main/README.rst Install the feedparser library using pip. This is the standard method for adding the package to your Python environment. ```console $ pip install feedparser ``` -------------------------------- ### HTTP Content-Type Header with Charset Source: https://github.com/kurtmckee/feedparser/blob/main/docs/character-encoding.md An example of an HTTP Content-Type header specifying the character encoding for a resource. ```text Content-Type: text/html; charset="utf-8" ``` -------------------------------- ### Rendered HTML Form Example Source: https://github.com/kurtmckee/feedparser/blob/main/docs/reference-feed-textinput.md This HTML snippet demonstrates how a feed's text input form might be rendered for user interaction. ```html
``` -------------------------------- ### XML Declaration with Encoding Source: https://github.com/kurtmckee/feedparser/blob/main/docs/character-encoding.md An example of an XML declaration specifying the document's encoding. ```xml ``` -------------------------------- ### Parse Feed from Local File Source: https://github.com/kurtmckee/feedparser/blob/main/docs/introduction.md Parse a feed from a local file path. Use the appropriate path syntax for your operating system. This example uses a Windows path. ```pycon >>> import feedparser >>> d = feedparser.parse(r'c:\incoming\atom10.xml') >>> d['feed']['title'] 'Sample Feed' ``` -------------------------------- ### Sample RSS Feed XML Source: https://github.com/kurtmckee/feedparser/blob/main/docs/common-rss-elements.md An example of an RSS 2.0 feed structure, illustrating common channel and item elements. ```xml Sample Feed For documentation only http://example.org/ Sat, 07 Sep 2002 00:00:01 GMT First entry title http://example.org/entry/3 Watch out for nasty tricks Thu, 05 Sep 2002 00:00:01 GMT http://example.org/entry/3 ``` -------------------------------- ### Embedding Javascript in HTML Source: https://github.com/kurtmckee/feedparser/blob/main/docs/html-sanitization.md This example demonstrates how malicious Javascript can be embedded within HTML using style attributes, even with naive sanitizers. ```html Watch out for <span style="background: url(javascript:window.location='http://example.org/')"> nasty tricks</span> ``` ```html Watch out for <span style="any: expression(window.location='http://example.org/')"> nasty tricks</span> ``` -------------------------------- ### Annotated RSS 2.0 Feed with Namespaces Source: https://github.com/kurtmckee/feedparser/blob/main/docs/annotated-rss20-dc.md This XML snippet represents an RSS 2.0 feed that incorporates multiple namespaces, including Dublin Core (dc), for richer metadata. It serves as an example for demonstrating how to parse and access namespaced elements. ```xml >> import feedparser >>> d = feedparser.parse('https://domain.example/examples/rss20.xml') >>> e = d.entries[0] >>> len(e.enclosures) 1 >>> e.enclosures[0] {'type': 'audio/mpeg', 'length': '1069871', 'href': 'http://example.org/audio/demo.mp3'} ``` -------------------------------- ### Basic Authentication (HTTPBasicAuthHandler) Source: https://github.com/kurtmckee/feedparser/blob/main/docs/http-authentication.md Download a feed protected by basic authentication using urllib2's HTTPBasicAuthHandler. This is a more secure method than embedding credentials in the URL. The handler can store credentials for multiple sites. ```python import urllib2, feedparser # Construct the authentication handler auth = urllib2.HTTPBasicAuthHandler() # Add password information: realm, host, user, password. # A single handler can contain passwords for multiple sites; # urllib2 will sort out which passwords get sent to which sites # based on the realm and host of the URL you're retrieving auth.add_password('BasicTest', 'feedparser.org', 'test', 'basic') # Pass the authentication handler to the feed parser. # handlers is a list because there might be more than one # type of handler (urllib2 defines lots of different ones, # and you can build your own) d = feedparser.parse( 'https://domain.example/examples/basic_auth.xml', handlers=[auth], ) ``` -------------------------------- ### Default Base URI from Content-Location Header Source: https://github.com/kurtmckee/feedparser/blob/main/docs/resolving-relative-links.md Shows how the Content-Location header provides the default base URI when no root-level xml:base is declared. This can still be overridden by child elements. ```pycon >>> import feedparser >>> d = feedparser.parse("https://domain.example/examples/http_base.xml") >>> d.feed.link 'http://example.org/index.html' >>> d.entries[0].link 'http://example.org/archives/000001.html' ``` -------------------------------- ### Accessing Detailed Atom Element Information Source: https://github.com/kurtmckee/feedparser/blob/main/docs/atom-detail.md Demonstrates how to parse an Atom feed and access detailed information for feed-level elements (title, subtitle, rights) and entry-level elements (title, summary, content). Use this to retrieve structured metadata like content type, base URI, language, and the actual value. ```python import feedparser d = feedparser.parse('https://domain.example/examples/atom10.xml') d.feed.title_detail d.feed.subtitle_detail d.feed.rights_detail d.entries[0].title_detail d.entries[0].summary_detail len(d.entries[0].content) d.entries[0].content[0] ``` -------------------------------- ### Access Atom Feed as Atom Feed Source: https://github.com/kurtmckee/feedparser/blob/main/docs/content-normalization.md Demonstrates accessing elements of an Atom feed using standard Atom terminology. Ensure the feed URL is correct. ```python >>> import feedparser >>> d = feedparser.parse('https://domain.example/examples/atom10.xml') >>> d['channel']['title'] 'Sample Feed' >>> d['channel']['link'] 'http://example.org/' >>> d['channel']['description'] 'For documentation only >>> len(d['items']) 1 >>> e = d['items'][0] >>> e['title'] 'First entry title' >>> e['link'] 'http://example.org/entry/3' >>> e['description'] 'Watch out for nasty tricks' >>> e['author'] 'Mark Pilgrim (mark@example.org)' ``` -------------------------------- ### Noticing Permanent Redirects (301) Source: https://github.com/kurtmckee/feedparser/blob/main/docs/http-redirect.md For permanent moves, check `d.status` for the 301 code and `d.href` for the new location. Update your stored feed URLs to `d.href` to avoid repeatedly requesting the old, now invalid, address. ```python >>> import feedparser >>> d = feedparser.parse('https://domain.example/examples/permanent.xml') >>> d.status 301 >>> d.href 'http://feedparser.org/docs/examples/atom10.xml' >>> d.feed.title 'Sample Feed' ``` -------------------------------- ### Using ETags to Reduce Bandwidth Source: https://github.com/kurtmckee/feedparser/blob/main/docs/http-etag.md Demonstrates how to fetch a feed, extract its ETag, and then use that ETag in a subsequent request to check if the feed has changed. A status code of 304 indicates no changes. ```python import feedparser d = feedparser.parse('https://domain.example/examples/atom10.xml') print(d.etag) d2 = feedparser.parse('https://domain.example/examples/atom10.xml', etag=d.etag) print(d2.status) print(d2.feed) print(d2.entries) print(d2.debug_message) ``` -------------------------------- ### Set Base URI with xml:base on Root Element Source: https://github.com/kurtmckee/feedparser/blob/main/docs/resolving-relative-links.md Demonstrates how an xml:base attribute on the root-level element sets the base URI for all URIs within the feed. ```pycon >>> import feedparser >>> d = feedparser.parse("https://domain.example/examples/base.xml") >>> d.feed.link 'http://example.org/index.html' >>> d.feed.generator_detail.href 'http://example.org/generator/' ``` -------------------------------- ### Accessing Feed Version Source: https://github.com/kurtmckee/feedparser/blob/main/docs/version-detection.md Demonstrates how to access the detected feed version after parsing different feed types. The 'version' attribute will contain a string indicating the detected format (e.g., 'atom10', 'rss20'). ```python >>> d = feedparser.parse('https://domain.example/examples/atom10.xml') >>> d.version 'atom10' >>> d = feedparser.parse('https://domain.example/examples/atom03.xml') >>> d.version 'atom03' >>> d = feedparser.parse('https://domain.example/examples/rss20.xml') >>> d.version 'rss20' >>> d = feedparser.parse('https://domain.example/examples/rss20dc.xml') >>> d.version 'rss20' >>> d = feedparser.parse('https://domain.example/examples/rss10.rdf') >>> d.version 'rss10' ``` -------------------------------- ### Parsing Feed with Non-Standard Prefixes Source: https://github.com/kurtmckee/feedparser/blob/main/docs/namespace-handling.md Demonstrates parsing an RDF feed with non-standard namespaces and accessing a namespaced element (prism:issn). Shows how feedparser maps prefixes to namespaces and raises an error for undefined prefixes. ```python >>> import feedparser >>> d = feedparser.parse('https://domain.example/examples/nonstandard_prefix.rdf') >>> d.feed.prism_issn '0028-0836' >>> d.feed.foo_issn Traceback (most recent call last): File "feedparser\util.py", line 149, in __getattr__ return self.__getitem__(key) File "feedparser\util.py", line 112, in __getitem__ return dict.__getitem__(self, key) KeyError: 'foo_issn' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "", line 1, in File "feedparser\util.py", line 151, in __getattr__ raise AttributeError("object has no attribute '%s'" % key) AttributeError: object has no attribute 'foo_issn' ``` ```python >>> d.namespaces {'': 'http://purl.org/rss/1.0/', 'prism': 'http://prismstandard.org/namespaces/1.2/basic/', 'rdf': 'http://www.w3.org/1999/02/22-rdf-syntax-ns#'} ``` -------------------------------- ### Test Feed Element Existence Source: https://github.com/kurtmckee/feedparser/blob/main/docs/basic-existence.md Use the 'in' operator to check if a feed element exists. Use the .get() method to retrieve an element's value with a default if it doesn't exist. This prevents errors when dealing with potentially missing feed data. ```pycon >>> import feedparser >>> d = feedparser.parse('https://domain.example/examples/atom10.xml') >>> 'title' in d.feed True >>> 'ttl' in d.feed False >>> d.feed.get('title', 'No title') 'Sample feed' >>> d.feed.get('ttl', 60) 60 ``` -------------------------------- ### Digest Authentication (Inline) Source: https://github.com/kurtmckee/feedparser/blob/main/docs/http-authentication.md Download a feed protected by digest authentication by embedding username and password directly in the URL. Feedparser automatically handles the upgrade from basic to digest authentication if needed, but this method is insecure. ```python import feedparser d = feedparser.parse('http://test:digest@feedparser.org/docs/examples/digest_auth.xml') d.feed.title ``` -------------------------------- ### Feed Image XML Structure Source: https://github.com/kurtmckee/feedparser/blob/main/docs/reference-feed-image.md This XML snippet demonstrates the structure of a feed image as it might appear in a feed. ```xml Feed logo http://example.org/logo.png http://example.org/ 80 15 Visit my home page ``` -------------------------------- ### Sample Atom Feed XML Source: https://github.com/kurtmckee/feedparser/blob/main/docs/common-atom-elements.md This is a sample Atom 1.0 feed in XML format. It includes common elements like title, subtitle, link, rights, id, generator, updated, and entry. ```xml Sample Feed For documentation <em>only</em> <p>Copyright 2005, Mark Pilgrim</p>< tag:feedparser.org,2005-11-09:/docs/examples/atom10.xml Sample Toolkit 2005-11-09T11:56:34Z First entry title tag:feedparser.org,2005-11-09:/docs/examples/atom10.xml:3 2005-11-09T00:23:47Z 2005-11-09T11:56:34Z Watch out for nasty tricks
Watch out for nasty tricks
``` -------------------------------- ### Set Custom User-Agent Source: https://github.com/kurtmckee/feedparser/blob/main/docs/http-useragent.md Set a custom User-Agent header for feedparser requests. This is recommended for applications to identify themselves to web servers. ```python import feedparser d = feedparser.parse( "https://domain.example/examples/atom10.xml", agent="MyApp/1.0 +http://domain.example/", ) ``` -------------------------------- ### Accessing Atom Feed Contributors Source: https://github.com/kurtmckee/feedparser/blob/main/docs/uncommon-atom.md Demonstrates how to access the list of contributors for an Atom feed entry. Each contributor is represented as a dictionary containing their name, href, and email. ```python >>> import feedparser >>> d = feedparser.parse('https://domain.example/examples/atom10.xml') >>> e = d.entries[0] >>> len(e.contributors) 2 >>> e.contributors[0] {'name': 'Joe', 'href': 'http://example.org/joe/', 'email': 'joe@example.org'} >>> e.contributors[1] {'name': 'Sam', 'href': 'http://example.org/sam/', 'email': 'sam@example.org'} ``` -------------------------------- ### Accessing Namespaced Elements with FeedParser Source: https://github.com/kurtmckee/feedparser/blob/main/docs/namespace-handling.md Demonstrates how to parse a feed and access a namespaced element (prism:issn) and the feed's namespaces. The prefix used for accessing elements is the namespace's preferred prefix, not necessarily the one in the feed. ```python >>> import feedparser >>> d = feedparser.parse('https://domain.example/examples/prism.rdf') >>> d.feed.prism_issn '0028-0836' >>> d.namespaces {'': 'http://purl.org/rss/1.0/', 'prism': 'http://prismstandard.org/namespaces/1.2/basic/', 'rdf': 'http://www.w3.org/1999/02/22-rdf-syntax-ns#'} ``` -------------------------------- ### Accessing Feed Cloud Source: https://github.com/kurtmckee/feedparser/blob/main/docs/uncommon-rss.md Retrieve information about the feed's cloud, used for real-time notifications. The structure includes domain, port, path, registerprocedure, and protocol. ```python >>> import feedparser >>> d = feedparser.parse('https://domain.example/examples/rss20.xml') >>> d.feed.cloud {'domain': 'rpc.example.com', 'port': '80', 'path': '/RPC2', 'registerprocedure': 'pingMe', 'protocol': 'soap'} ``` -------------------------------- ### Access RSS Feed as Atom Feed Source: https://github.com/kurtmckee/feedparser/blob/main/docs/content-normalization.md Demonstrates accessing elements of an RSS feed as if it were an Atom feed, showcasing Universal Feed Parser's normalization capabilities. Ensure the feed URL is correct. ```python >>> import feedparser >>> d = feedparser.parse('https://domain.example/examples/rss20.xml') >>> d.feed.subtitle_detail {'type': 'text/html', 'base': 'http://feedparser.org/docs/examples/rss20.xml', 'language': None, 'value': 'For documentation only'} >>> len(d.entries) 1 >>> e = d.entries[0] >>> e.links [{'rel': 'alternate', 'type': 'text/html', 'href': 'http://example.org/item/1'}] >>> e.summary_detail {'type': 'text/html', 'base': 'http://feedparser.org/docs/examples/rss20.xml', 'language': 'en', 'value': 'Watch out for nasty tricks'} >>> e.updated_parsed (2002, 9, 5, 0, 0, 1, 3, 248, 0) ``` -------------------------------- ### Annotated RSS 1.0 Feed Source: https://github.com/kurtmckee/feedparser/blob/main/docs/annotated-rss10.md This is a sample RSS 1.0 feed. Each element is annotated with a link to further documentation, demonstrating how to access specific data points after parsing. ```xml Sample Feed http://www.example.org/ For documentation only en Mark Pilgrim (mark@example.org) 2004-06-04T17:40:33-05:00 First of all http://example.org/archives/2002/09/04.html#first_of_all Americans are fat. Smokers are stupid. People who don’t speak Perl are irrelevant. Quotes 2004-05-30T14:23:54-06:00 Ian Hickson: Americans are fat. Smokers are stupid. People who don’t speak Perl are irrelevant. ]]> ``` -------------------------------- ### Annotated RSS 2.0 Feed Structure Source: https://github.com/kurtmckee/feedparser/blob/main/docs/annotated-rss20.md This is a sample RSS 2.0 feed with annotations. Each annotated value links to documentation explaining how to access it after parsing. ```xml Sample Feed For documentation only http://example.org/ en Copyright 2004, Mark Pilgrim editor@example.org webmaster@example.org Sat, 07 Sep 2002 0:00:01 GMT Examples Sample Toolkit http://feedvalidator.org/docs/rss2.html 60 http://example.org/banner.png Example banner http://example.org/ 80 15 Search Search this site: q http://example.org/mt/mt-search.cgi First item title http://example.org/item/1 Watch out for <span style="background: url(javascript:window.location=’http://example.org/’)”> nasty tricks</span> mark@example.org Miscellaneous http://example.org/comments/1 http://example.org/guid/1 Thu, 05 Sep 2002 0:00:01 GMT ```