# Streamlit Documentation Website ## Introduction The Streamlit Documentation Website is a Next.js-based static site generator that powers docs.streamlit.io, the official documentation portal for Streamlit. Built with Next.js 14, the project leverages MDX for content authoring, enabling technical writers to embed React components directly within Markdown files. The architecture emphasizes automation and scalability, featuring Python-based docstring extraction that introspects Streamlit's source code to generate API documentation, and multi-version support that creates separate documentation paths for each Streamlit release. The system integrates several key technologies: Algolia for search functionality, Ghost CMS for blog content, Netlify for deployment and CDN distribution, and a comprehensive component library for rich documentation features. The project follows a static-first approach where all pages are pre-rendered at build time, ensuring fast page loads and excellent SEO performance. Content is organized in a hierarchical menu structure defined in a single source of truth (`menu.md`), with automatic breadcrumb generation, version management, and navigation between related pages. ## Build and Deployment ### Development Server ```bash # Install dependencies make # Start development server at http://localhost:3000 make up ``` ### Production Build ```bash # Generate static site (exports to /out directory) make export # Build includes: # - All markdown content compiled to static HTML # - Docstring JSON embedded in pages # - Version-specific routes generated # - Sitemap and robots.txt created ``` ### Docstring Generation ```bash # Build Docker image and generate API documentation from Streamlit source make docstrings # Process: # 1. Builds Docker image with specific Streamlit version # 2. Introspects Streamlit source code # 3. Extracts function signatures, parameters, docstrings # 4. Converts RST/NumPy docstrings to HTML # 5. Updates python/streamlit.json with new version data ``` ```python # python/generate.py - Core docstring extraction import streamlit as st import inspect from numpydoc.docscrape import NumpyDocString import docstring_parser def get_streamlit_docstring_dict(): """Generate complete API documentation for all Streamlit objects""" # Define all Streamlit objects to document obj_key = { streamlit: ["streamlit", "st"], streamlit.cache_data: ["streamlit.cache_data", "st.cache_data"], streamlit.delta_generator.DeltaGenerator: ["streamlit.delta_generator.DeltaGenerator"], } module_docstring_dict = {} for obj, key in obj_key.items(): module_docstring_dict.update( get_obj_docstring_dict(obj, *key) ) return module_docstring_dict # Output format: python/streamlit.json { "1.40.0": { "streamlit.write": { "name": "write", "signature": "st.write(*args, **kwargs)", "description": "Write arguments to the app...", "args": [ { "name": "args", "type_name": "any", "description": "One or more values to write", "is_optional": false } ], "examples": "st.write('Hello', 'World')", "source": "https://github.com/streamlit/streamlit/blob/1.40.0/lib/streamlit/write.py#L42" } } } ``` ### Search Index Update ```bash # Rebuild Algolia search index make search ``` ```javascript // scripts/build-search-index.js const algoliasearch = require('algoliasearch'); const { convert } = require('html-to-text'); const fs = require('fs'); const client = algoliasearch('XNXFGO6BQ1', process.env.ALGOLIA_ADMIN_KEY); const index = client.initIndex('documentation'); // Parse all built HTML files const pages = getAllFilesInDirectory('.next/server/pages'); const records = pages.map(page => { const html = fs.readFileSync(page, 'utf8'); return { objectID: extractUrl(page), title: extractTitle(html), content: convert(html, { wordwrap: false }), url: extractUrl(page), category: extractCategory(html), version: extractVersion(page), icon: metadata.icon, color: metadata.color }; }); // Upload to Algolia await index.saveObjects(records); ``` ## Content Management System ### Markdown File Structure ```markdown --- title: Create a Component slug: /develop/concepts/custom-components/create --- # Create a Component This is the page content with **markdown** formatting. Callouts use MDX components with blank lines after opening tags. ``` ### Loading and Processing Content ```javascript // lib/node/api.js import fs from 'fs'; import matter from 'gray-matter'; import path from 'path'; export function getArticleBySlug(slug, fields = []) { const realSlug = slug.replace(/\.md$/, ''); const fullPath = path.join(articlesDirectory, `${realSlug}.md`); const fileContents = fs.readFileSync(fullPath, 'utf8'); const { data, content } = matter(fileContents); const items = {}; fields.forEach((field) => { if (field === 'slug') { items[field] = realSlug; } if (field === 'content') { items[field] = content; } if (data[field]) { items[field] = data[field]; } }); return items; } // Usage in getStaticProps const article = getArticleBySlug('/develop/api-reference/charts/bar-chart', [ 'title', 'slug', 'content', 'description' ]); // Returns: { title: "st.bar_chart", slug: "/develop/...", content: "...", description: "..." } ``` ### Menu System ```javascript // lib/node/api.js - Menu structure from menu.md export function getMenu() { const fileContents = fs.readFileSync('content/menu.md', 'utf8'); const { data } = matter(fileContents); const flatMenu = data.site_menu; // Convert flat array to nested tree const menu = []; const stack = [{ children: menu, depth: -1 }]; flatMenu.forEach(item => { const depth = item.category.split(' / ').length - 1; const menuItem = { name: item.category.split(' / ').pop(), url: item.url, visible: item.visible !== false, children: [], depth: depth }; // Pop stack to find parent while (stack[stack.length - 1].depth >= depth) { stack.pop(); } stack[stack.length - 1].children.push(menuItem); stack.push(menuItem); }); return menu; } ``` ```yaml # content/menu.md - Single source of truth for navigation --- site_menu: - category: Get Started url: /get-started visible: true - category: Get Started / Installation url: /get-started/installation visible: true - category: Get Started / Installation / Quickstart url: /get-started/installation/quickstart visible: true - category: Develop url: /develop visible: true - category: Develop / API Reference url: /develop/api-reference visible: true - category: Develop / API Reference / Charts / st.bar_chart url: /develop/api-reference/charts/st.bar_chart visible: true --- ``` ## Dynamic Page Rendering ### Catch-all Route Handler ```javascript // pages/[...slug].js - Main page renderer import { serialize } from 'next-mdx-remote/serialize'; import { MDXRemote } from 'next-mdx-remote'; import matter from 'gray-matter'; export async function getStaticPaths() { const articles = getArticleSlugs(); // Recursively find all .md files const paths = []; articles.forEach(article => { const fileContents = fs.readFileSync(article, 'utf8'); const { data, content } = matter(fileContents); // Base path from frontmatter slug paths.push({ params: { slug: data.slug.split('/').filter(Boolean) } }); // If page uses , generate versioned paths if (/ { ['oss', 'sis', 'na'].forEach(platform => { const versionSlug = getVersionAndPlatformString(version, platform); if (versionSlug) { paths.push({ params: { slug: [versionSlug, ...data.slug.split('/').filter(Boolean)] } }); } }); }); } }); return { paths, fallback: false }; } export async function getStaticProps({ params }) { const slug = params.slug.join('/'); // Extract version from URL if present const [version, platform] = getVersionAndPlatformFromPathPart(params.slug[0]); // Find markdown file const filename = findFileForSlug(slug); const fileContents = fs.readFileSync(filename, 'utf8'); const { data, content } = matter(fileContents); // Load docstrings for this version const docstrings = {}; if (/ { docstrings[func] = DOCSTRINGS[version || LATEST_VERSION][func]; }); } // Serialize MDX with plugins const mdxSource = await serialize(content, { mdxOptions: { rehypePlugins: [ rehypeSlug, // Add IDs to headings rehypeAutolinkHeadings // Add anchor links to headings ], remarkPlugins: [ remarkUnwrapImages, // Remove

wrapper around images remarkGfm // GitHub Flavored Markdown ] } }); // Get menu and navigation const menu = getMenu(); const { prev, next } = getPreviousNextFromMenu(menu, data.slug); return { props: { data, source: mdxSource, docstrings, menu, prev, next, versionFromSlug: version, platformFromSlug: platform } }; } export default function Article({ data, source, docstrings, menu, prev, next }) { // MDX component mapping const components = { // Callouts Note, Tip, Warning, Important, // Content blocks Code, Image, YouTube, // Layouts Flex, Masonry, TileContainer, // API documentation Autofunction: (props) => ( ), // Override HTML elements h1: H1, h2: H2, h3: H3, pre: Code, a: ({ href, children }) => {children} }; return (

{data.title}

{/* Table of contents */} ); } ``` ### MDX Component Usage in Content ```markdown --- title: st.bar_chart slug: /develop/api-reference/charts/st.bar_chart --- # st.bar_chart ## Example ```python import streamlit as st import pandas as pd df = pd.DataFrame({ 'col1': [1, 2, 3, 4], 'col2': [10, 20, 30, 40] }) st.bar_chart(df) ``` Bar charts are best for comparing categorical data. Large datasets may impact performance. ``` ## Version Management System ### Version Context Provider ```javascript // lib/next/VersionContext.jsx import { createContext, useContext, useState } from 'react'; const VersionContext = createContext(); export function VersionContextProvider({ versionFromSlug, platformFromSlug, children }) { const [version, setVersion] = useState(versionFromSlug || 'latest'); const [platform, setPlatform] = useState(platformFromSlug || 'oss'); const goToLatest = () => setVersion('latest'); const goToOpenSource = () => setPlatform('oss'); return ( {children} ); } export function useVersionContext() { const context = useContext(VersionContext); if (!context) { throw new Error('useVersionContext must be used within VersionContextProvider'); } return context; } // Parse version string from URL export function getVersionAndPlatformFromPathPart(pathPart) { // "1.40.0-sis" -> ["1.40.0", "sis"] // "latest" -> ["latest", "oss"] // "1.40.0" -> ["1.40.0", "oss"] if (!pathPart || !looksLikeVersionString(pathPart)) { return [null, null]; } const parts = pathPart.split('-'); const version = parts[0]; const platform = parts[1] || 'oss'; return [version, platform]; } // Generate URL path segment export function getVersionAndPlatformString(version, platform) { // Return null for defaults (omitted from URL) if (version === 'latest' && platform === 'oss') { return null; } if (platform === 'oss') { return version; } return `${version}-${platform}`; } // Usage in component function VersionSelector() { const { version, setVersion } = useVersionContext(); return ( ); } ``` ### Versioned URL Structure ```javascript // next.config.mjs - URL rewrites for version handling export default { async rewrites() { return [ // Alias /sis to specific SIS version { source: "/sis/:path*", destination: "/1.42.0/:path*" }, // Alias /latest to base URL (latest is default) { source: `/latest/:path*`, destination: "/:path*" }, // Specific version number also aliases to base { source: `/${LATEST_VERSION}/:path*`, destination: "/:path*" } ]; } }; // URL patterns generated: // /develop/api-reference/charts/st.bar_chart (latest, oss - default) // /1.39.0/develop/api-reference/charts/st.bar_chart (version 1.39.0, oss) // /1.42.0-sis/develop/api-reference/charts/st.bar_chart (version 1.42.0, sis platform) // /latest-na/develop/api-reference/charts/st.bar_chart (latest, na platform) ``` ## API Documentation Components ### Autofunction Component ```javascript // components/blocks/autofunction.js import { useVersionContext } from '../../lib/next/VersionContext'; import Prism from 'prismjs'; export default function Autofunction({ function: functionName, // e.g., "streamlit.write" docstrings, // Passed from getStaticProps oldName // For deprecated functions }) { const { version, setVersion } = useVersionContext(); const [activeTab, setActiveTab] = useState('description'); // Get docstring for current version const doc = docstrings[functionName]; if (!doc) { return
Documentation not available for {functionName}
; } return (
{/* Version selector */} {/* Function signature */}
{doc.signature} View source
{/* Deprecation warning */} {doc.deprecated && ( Deprecated: {doc.deprecated.message} {oldName && Use {oldName} instead.} )} {/* Tabs */}
{/* Description tab */} {activeTab === 'description' && (
{/* Parameters */} {doc.args && doc.args.length > 0 && (

Parameters

{doc.args.map(arg => (
{arg.name} {arg.type_name && ({arg.type_name})} {arg.is_optional && optional}
{arg.default && (
Default: {arg.default}
)}
))}
)} {/* Returns */} {doc.returns && (

Returns

)}
)} {/* Examples tab */} {activeTab === 'examples' && doc.examples && (
)}
); } ``` ### API Reference Landing Page ```markdown --- title: API Reference slug: /develop/api-reference --- # API Reference ## Chart Elements Bar chart example #### Bar chart Display a bar chart. ```python st.bar_chart(data) ``` Line chart example #### Line chart Display a line chart. ```python st.line_chart(data) ``` ``` ## Navigation System ### Breadcrumb Generation ```javascript // lib/purejs/breadcrumbHelpers.js export function breadcrumbsForSlug(menu, slugStr, path = []) { for (const item of menu) { const newPath = [...path, item]; // Found matching item if (item.url === slugStr) { return newPath.map(pathItem => ({ name: pathItem.name, url: pathItem.url, icon: pathItem.icon, color: pathItem.color })); } // Search children if (item.children && item.children.length > 0) { const result = breadcrumbsForSlug(item.children, slugStr, newPath); if (result) return result; } } return null; } // Usage in component function BreadCrumbs({ slug, menu }) { const breadcrumbs = breadcrumbsForSlug(menu, slug); return ( ); } ``` ### Previous/Next Navigation ```javascript // lib/next/utils.js export function getPreviousNextFromMenu(menu, slugStr, parent = null) { let prev = null; let current = null; let next = null; let found = false; for (let i = 0; i < menu.length; i++) { const item = menu[i]; // Skip invisible items if (item.visible === false) continue; // Skip dividers if (item.type === 'divider') continue; if (found && !next) { next = item; break; } if (item.url === slugStr) { current = item; found = true; continue; } // Search children recursively if (item.children && item.children.length > 0) { const result = getPreviousNextFromMenu(item.children, slugStr, item); if (result.current) { return result; } } if (!found) { prev = item; } } return { current, prev, next }; } // Usage in page function ArrowLinkContainer({ prev, next }) { return (
{prev && ( ← Previous {prev.name} )} {next && ( Next → {next.name} )}
); } ``` ### Table of Contents (Floating Nav) ```javascript // components/utilities/floatingNav.js import { useEffect, useState } from 'react'; export default function FloatingNav() { const [headings, setHeadings] = useState([]); const [activeId, setActiveId] = useState(''); useEffect(() => { // Extract H2 and H3 headings from page const elements = Array.from( document.querySelectorAll('article h2, article h3') ); const headingsList = elements.map(elem => ({ id: elem.id, text: elem.textContent, level: elem.tagName })); setHeadings(headingsList); // Track scroll position for active heading const observer = new IntersectionObserver( (entries) => { entries.forEach(entry => { if (entry.isIntersecting) { setActiveId(entry.target.id); } }); }, { rootMargin: '-100px 0px -66%' } ); elements.forEach(elem => observer.observe(elem)); return () => observer.disconnect(); }, []); return ( ); } ``` ## Search Integration ### Algolia Configuration ```javascript // components/utilities/search.js import algoliasearch from 'algoliasearch/lite'; import { InstantSearch, SearchBox, Hits, Configure } from 'react-instantsearch-dom'; import { useEffect, useState } from 'react'; const searchClient = algoliasearch( 'XNXFGO6BQ1', 'dd82fd414de60db7ffbcbf30eca68013' // Public search-only key ); export default function Search() { const [isOpen, setIsOpen] = useState(false); const { version } = useVersionContext(); // Keyboard shortcut (Cmd/Ctrl+K) useEffect(() => { const handleKeyDown = (e) => { if ((e.metaKey || e.ctrlKey) && e.key === 'k') { e.preventDefault(); setIsOpen(true); } if (e.key === 'Escape') { setIsOpen(false); } }; window.addEventListener('keydown', handleKeyDown); return () => window.removeEventListener('keydown', handleKeyDown); }, []); return (
); } function Hit({ hit }) { return (
{hit.category}
{hit.title}
 ); } ``` ### Keyboard Navigation ```javascript // components/utilities/search.js - Extended function SearchResults({ hits }) { const [selectedIndex, setSelectedIndex] = useState(0); useEffect(() => { const handleKeyDown = (e) => { switch (e.key) { case 'ArrowDown': e.preventDefault(); setSelectedIndex(i => Math.min(i + 1, hits.length - 1)); break; case 'ArrowUp': e.preventDefault(); setSelectedIndex(i => Math.max(i - 1, 0)); break; case 'Enter': e.preventDefault(); if (hits[selectedIndex]) { window.location.href = hits[selectedIndex].url; } break; } }; window.addEventListener('keydown', handleKeyDown); return () => window.removeEventListener('keydown', handleKeyDown); }, [hits, selectedIndex]); return (
{hits.map((hit, index) => ( ))}
); } ``` ## Configuration and Security ### Next.js Configuration ```javascript // next.config.mjs import { DOCSTRINGS, VERSIONS_LIST, LATEST_VERSION, PLATFORM_VERSIONS } from './lib/node/defaults.js'; export default { // Make docstrings and versions available to pages serverRuntimeConfig: { DOCSTRINGS, // Full docstring JSON (server-side only) VERSIONS_LIST, // ["1.40.0", "1.39.0", ...] LATEST_VERSION, // "1.40.0" PLATFORM_VERSIONS, // { sis: [...], na: [...] } }, // Public config (available in browser) publicRuntimeConfig: { VERSIONS_LIST, LATEST_VERSION, PLATFORM_VERSIONS, }, // Static export mode output: 'export', // Custom webpack configuration webpack: (config) => { config.resolve.fallback = { fs: false }; // Markdown loader config.module.rules.push({ test: /\.md$/, use: 'frontmatter-markdown-loader' }); // SVG loader config.module.rules.push({ test: /\.svg$/, use: ['@svgr/webpack', 'file-loader'] }); return config; } }; ``` ### Content Security Policy ```javascript // next.config.mjs - Security headers const CSP_HEADER = [ "upgrade-insecure-requests;", "default-src 'none';", "font-src 'self';", "form-action 'self';", "script-src", "'self'", "'unsafe-inline'", // NextJS inline scripts "'unsafe-eval'", // Required for MDXRemote "https://cdn.segment.com/", "https://widget.kapa.ai/", "https://*.algolia.net/", ";", "connect-src", "'self'", "https://*.streamlit.app/", "https://streamlit.ghost.io/", "https://api.segment.io/", "https://*.algolia.net/", ";", "img-src 'self' data: https:;", "style-src 'self' 'unsafe-inline';", "frame-src https:;", ]; export default { async headers() { return [{ source: '/(.*)', headers: [ { key: 'Content-Security-Policy', value: CSP_HEADER.join(' ') }, { key: 'Strict-Transport-Security', value: 'max-age=31536000; includeSubDomains' }, { key: 'X-Content-Type-Options', value: 'nosniff' } ] }]; } }; ``` ### Netlify Deployment ```toml # netlify.toml [build] command = "make export" publish = "out" [[plugins]] package = "@netlify/plugin-nextjs" [[headers]] for = "/*" [headers.values] Strict-Transport-Security = "max-age=31536000; includeSubDomains" X-Content-Type-Options = "nosniff" Content-Security-Policy = """ upgrade-insecure-requests; \ default-src 'none'; \ script-src 'self' 'unsafe-inline' 'unsafe-eval' https://cdn.segment.com/; \ connect-src 'self' https://*.algolia.net/ https://api.segment.io/; \ img-src 'self' data: https:; \ style-src 'self' 'unsafe-inline'; \ """ ``` ## Component Library ### Callout Components ```jsx // components/blocks/note.js export default function Note({ children }) { return (
ℹ️
{children}
); } // components/blocks/warning.js export default function Warning({ children }) { return (
⚠️
{children}
); } // Usage in markdown This is informational content. This is a warning message. ``` ### Layout Components ```jsx // components/layouts/flex.js export default function Flex({ children, gap = '1rem' }) { return (
{children}
); } // components/layouts/tileContainer.js export default function TileContainer({ children, layout = 'grid' }) { return (
{children}
); } // Usage in markdown Content 1 Content 2 ``` ### Code Highlighting ```jsx // components/blocks/code.js import Prism from 'prismjs'; import { useEffect, useRef } from 'react'; export default function Code({ children, language = 'python' }) { const codeRef = useRef(null); useEffect(() => { if (codeRef.current) { Prism.highlightElement(codeRef.current); } }, [children]); const handleCopy = () => { navigator.clipboard.writeText(children); }; return (
{language} 
        
          {children}
); } ``` ## Summary The Streamlit Documentation Website represents a sophisticated static site architecture that balances developer experience with content authoring simplicity. The system's core strength lies in its automation capabilities: Python scripts introspect Streamlit's source code to generate comprehensive API documentation, while Next.js's static generation produces fast-loading pages with excellent SEO. Content authors work exclusively in Markdown files with embedded MDX components, never needing to touch the JavaScript codebase, while the hierarchical menu system ensures consistent navigation across the entire site. Integration patterns demonstrate production-ready architectural decisions: version management through React Context and URL-based routing enables seamless documentation for multiple Streamlit releases, Algolia provides powerful search with faceting by version, and comprehensive Content Security Policies protect users while allowing necessary third-party integrations. The deployment pipeline via Netlify with CDN distribution, combined with automated docstring generation through Docker, creates a maintainable and scalable documentation platform that serves thousands of developers while remaining easy to contribute to and extend.