TagSoup

https://hackage.haskell.org/package/tagsoup/docs
Admin
TagSoup is a Haskell library for parsing and extracting information from well-formed XML or...
Tokens:1,927
Snippets:23
Trust Score:9.7
Update:1 month ago
Show doc for...
Tokens:
Raw
### Full paper extraction example

Source: https://hackage.haskell.org/package/tagsoup

A complete program that fetches a URL, parses the HTML, and extracts paper titles using TagSoup.

```haskell
module Main where

import Network.HTTP
import Text.HTML.TagSoup

openURL :: String -> IO String
openURL x = getResponseBody =<< simpleHTTP (getRequest x)

spjPapers :: IO ()
spjPapers = do
        tags <- parseTags <$> openURL "http://research.microsoft.com/en-us/people/simonpj/"
        let links = map f $ sections (~== "<A>") $
                    takeWhile (~/= "<a name=haskell>") $
                    drop 5 $ dropWhile (~/= "<a name=current>") tags
        putStr $ unlines links
    where
        f :: [Tag String] -> String
        f = dequote . unwords . words . fromTagText . head . filter isTagText

        dequote ('\"':xs) | last xs == '\"' = init xs
        dequote x = x

main :: IO ()
main = spjPapers
```

--------------------------------

### HTML structure for last modified date

Source: https://hackage.haskell.org/package/tagsoup

Example HTML fragment showing the target element containing the last modified date on the Haskell Wiki.

```html
<ul id="f-list">
  <li id="lastmod"> This page was last modified on 9 September 2013, at 22:38.</li>
  <li id="copyright">Recent content is available under <a href="/HaskellWiki:Copyrights" title="HaskellWiki:Copyrights">a simple permissive license</a>.</li>
  <li id="privacy"><a href="/HaskellWiki:Privacy_policy" title="HaskellWiki:Privacy_policy">Privacy policy</a></li>
  <li id="about"><a href="/HaskellWiki:About" title="HaskellWiki:About">About HaskellWiki</a></li>
  <li id="disclaimer"><a href="/HaskellWiki:General_disclaimer" title="HaskellWiki:General_disclaimer">Disclaimers</a></li>
</ul>
```

--------------------------------

### Download web page with HTTP package

Source: https://hackage.haskell.org/package/tagsoup

A basic Haskell program to download a web page's content and save it to a local file for inspection.

```haskell
module Main where

import Network.HTTP

openURL :: String -> IO String
openURL x = getResponseBody =<< simpleHTTP (getRequest x)

main :: IO ()
main = do
    src <- openURL "http://wiki.haskell.org/Haskell"
    writeFile "temp.htm" src
```

--------------------------------

### Fetch current time from a website using TagSoup

Source: https://hackage.haskell.org/package/tagsoup

Retrieves the current time from a specific HTML element on a webpage.

```haskell
module Main where

import Network.HTTP
import Text.HTML.TagSoup

openURL :: String -> IO String
openURL x = getResponseBody =<< simpleHTTP (getRequest x)

currentTime :: IO ()
currentTime = do
    tags <- parseTags <$> openURL "http://www.timeanddate.com/worldclock/uk/london"
    let time = fromTagText (dropWhile (~/= "<span id=ct>") tags !! 1)
    putStrLn time

main :: IO ()
main = currentTime
```

--------------------------------

### Extract paper titles from a website using TagSoup

Source: https://hackage.haskell.org/package/tagsoup

Fetches a list of papers from a URL and extracts titles by parsing HTML tags.

```haskell
module Main where

import Network.HTTP
import Text.HTML.TagSoup

openURL :: String -> IO String
openURL x = getResponseBody =<< simpleHTTP (getRequest x)

ndmPapers :: IO ()
ndmPapers = do
        tags <- parseTags <$> openURL "http://community.haskell.org/~ndm/downloads/"
        let papers = map f $ sections (~== "<li class=paper>") tags
        putStr $ unlines papers
    where
        f :: [Tag String] -> String
        f xs = fromTagText (xs !! 2)

main :: IO ()
main = ndmPapers
```

--------------------------------

### Finding hyperlinks in a section

Source: https://hackage.haskell.org/package/tagsoup

Selects all anchor tags within a section and maps them using a processing function.

```haskell
map f $ sections (~== "<A>") $ ...
```

--------------------------------

### Extracting Last Modified Date from a Wiki Page

Source: https://hackage.haskell.org/package/tagsoup

Fetches a webpage and uses TagSoup to locate a specific list item by its ID, then processes the inner text to extract a date string.

```haskell
module Main where

import Data.Char
import Network.HTTP
import Text.HTML.TagSoup

openURL :: String -> IO String
openURL x = getResponseBody =<< simpleHTTP (getRequest x)

haskellLastModifiedDateTime :: IO ()
haskellLastModifiedDateTime = do
    src <- openURL "http://wiki.haskell.org/Haskell"
    let lastModifiedDateTime = fromFooter $ parseTags src
    putStrLn $ "wiki.haskell.org was last modified on " ++ lastModifiedDateTime
    where fromFooter = unwords . drop 6 . words . innerText . take 2 . dropWhile (~/= "<li id=lastmod>")

main :: IO ()
main = haskellLastModifiedDateTime
```

--------------------------------

### Extracting a section of tags

Source: https://hackage.haskell.org/package/tagsoup

Uses takeWhile and dropWhile to isolate a specific portion of the tag list between two named anchors.

```haskell
takeWhile (~/= "<a name=haskell>") $
drop 5 $ dropWhile (~/= "<a name=current>") tags
```

--------------------------------

### Processing link text

Source: https://hackage.haskell.org/package/tagsoup

Extracts and cleans text content from a list of tags, handling whitespace and optional quotes.

```haskell
f = dequote . unwords . words . fromTagText . head . filter isTagText
```
Tokens:
Raw