### Schema Tool Definition Setup Source: https://answerdotai.github.io/toolslm/funccall.html Example setup for a tool definition using dict2obj. ```python tool = dict2obj(dict( description='Find real-…', inputSchema={ '$schema': 'http://json-schema.org/draft-07/schema#', **dict( additionalProperties=False, properties=dict( language=dict(description='Filter by …', items={'type': 'string'}, type='array'), matchCase=dict(default=False, description='Whether th…', type='boolean'), path=dict(description='Filter by …', type='string'), query=dict(description='The litera…', type='string'), useRegexp=dict(default=False, description='Whether to…', type='boolean')), required=['query'], type='object')}) ``` ```python props, req = tool.inputSchema['properties'], tool.inputSchema['required'] list(props) ``` ```python props.matchCase ``` ```python tool_hy = dict2obj(dict(name='run', description='Run command', inputSchema=dict(type='object', properties={'cmd': {'type': 'string'}, 'approval-policy': {'type': 'string', 'default': 'never'}}, required=['cmd']))) sig = schema2sig(tool_hy) test_eq(list(sig.parameters), ['cmd', 'approval_policy']) test_eq(sig.parameters['approval_policy'].default, 'never') test_eq(sig.parameters['approval_policy'].kind, Parameter.KEYWORD_ONLY) ``` ```python tool_req_hy = dict2obj(dict(name='run', description='Run command', inputSchema=dict(type='object', properties={'approval-policy': {'type': 'string'}, 'cmd': {'type': 'string'}}, required=['approval-policy', 'cmd']))) ``` ```python tool_collision = dict2obj(dict(name='run', description='Run command', inputSchema=dict(type='object', properties={'approval-policy': {'type': 'string'}, 'approval_policy': {'type': 'string'}}))) test_fail(lambda: schema2sig(tool_collision), contains='collision') ``` ```python test_eq(_py_nm(''), '_') ``` -------------------------------- ### Install toolslm Source: https://answerdotai.github.io/toolslm/index.html.md Install the library using pip. ```sh pip install toolslm ``` -------------------------------- ### Example URL for fastlite Source: https://answerdotai.github.io/toolslm/download.html.md An example URL pointing to the fastlite project, likely used for testing or demonstration purposes with the find_docs function. ```python fl_url = 'https://answerdotai.github.io/fastlite' ``` -------------------------------- ### Execute shell commands Source: https://answerdotai.github.io/toolslm/shell.html.md Examples of running code within the initialized shell instance. ```python shell = get_shell() ``` ```python r = shell.run_cell('print(3); 1+1') r.result,r.stdout ``` ```python r = shell.run_cell('raise Exception("blah")') print(exception2str(r.error_in_exec)) ``` ```python r = shell.run_cell('import time; time.sleep(10)', timeout=1) r.error_in_exec ``` -------------------------------- ### Create a doctype Tuple Source: https://answerdotai.github.io/toolslm/xml.html.md Usage example for creating a doctype tuple from content. ```python doc = 'This is a "sample"' mk_doctype(doc) ``` -------------------------------- ### Get GitHub repo file sizes with type filtering Source: https://answerdotai.github.io/toolslm/xml.html.md This example shows how to use repo2ctx to obtain a dictionary of file sizes from a GitHub repository. It filters for Python files and excludes files starting with an underscore, returning only the file names and their sizes. ```python print(repo2ctx('answerdotai/toolslm', types='py', skip_file_re='^_', out=False, files_only=True)) ``` -------------------------------- ### Execute shell commands Source: https://answerdotai.github.io/toolslm/shell.html Examples of running code within the shell, handling output, exceptions, and timeouts. ```python r = shell.run_cell('print(3); 1+1') r.result,r.stdout ``` ```python r = shell.run_cell('raise Exception("blah")') print(exception2str(r.error_in_exec)) ``` ```python r = shell.run_cell('import time; time.sleep(10)', timeout=1) r.error_in_exec ``` -------------------------------- ### Execute code with python function Source: https://answerdotai.github.io/toolslm/funccall.html Examples of executing Python code, handling timeouts, and managing namespaces. ```python python("""def factorial(n): if n == 0 or n == 1: return 1 else: return n * factorial(n-1) factorial(5)""") ``` ```python print(python('import time; time.sleep(10)', timeout=1)) ``` ```python python("a=1") a ``` ```python glb = {} python("a=3", glb=glb) a, glb['a'] ``` -------------------------------- ### Generate schema for get_schema Source: https://answerdotai.github.io/toolslm/funccall.html.md Example of generating a schema for the get_schema function itself. ```python get_schema(get_schema) ``` -------------------------------- ### Generate XML for Multiple Documents Source: https://answerdotai.github.io/toolslm/xml.html.md Example of generating a full XML string for a list of documents. ```python docs = [doc, 'And another one'] srcs = [None, 'doc.txt'] print(docs_xml(docs, srcs)) ``` -------------------------------- ### Convert GitHub repo to XML context with file type and size limits Source: https://answerdotai.github.io/toolslm/xml.html.md This example demonstrates converting a GitHub repository to XML context, filtering by specific file extensions (ipynb, py) and applying a maximum total output size. It also skips files matching a regular expression and disables the inclusion of notebook cell outputs. ```python print(repo2ctx('answerdotai/toolslm', exts=('ipynb','py'), skip_file_re='^_', out=False, max_total=500)) ``` -------------------------------- ### Call find_docs with fastlite URL Source: https://answerdotai.github.io/toolslm/download.html.md Demonstrates calling the find_docs function with the example fastlite URL to retrieve its documentation. ```python find_docs(fl_url) ``` -------------------------------- ### Generate XML for a Single Document Source: https://answerdotai.github.io/toolslm/xml.html.md Example of generating XML for one document using mk_doc. ```python mk_doc(1, doc, title="test") ``` -------------------------------- ### symslice Source: https://answerdotai.github.io/toolslm/inspecttools.html.md Returns the contents of the symbol from the given start to the end. ```APIDOC ## symslice ### Description Returns the contents of the symbol from the given start to the end. ### Parameters #### Request Body - **sym** (str) - Required - Dotted symbol path or "_last" for previous result - **start** (int) - Required - Starting index for slice - **end** (int) - Required - Ending index for slice ``` -------------------------------- ### Get Source File Path for Symbol Source: https://answerdotai.github.io/toolslm/xml.html.md Returns an MD string containing the filepath and contents for a given symbol's source file. The example shows how to use it with the `Path` object. ```python def sym2file( sym ): ``` ```python # print(sym2file(Path)) ``` -------------------------------- ### Get Schema for a Class Method Source: https://answerdotai.github.io/toolslm/funccall.html Illustrates retrieving the schema for an instance method of a class. Default values for parameters are correctly parsed. ```python class Dummy: def sums( self, a:int, # First thing to sum b:int=1 # Second thing to sum ): # The sum of the inputs "Adds a + b." print(f"Finding the sum of {a} and {b}") return a + b get_schema(Dummy.sums) ``` -------------------------------- ### Convert GitHub repo subdirectory to context Source: https://answerdotai.github.io/toolslm/xml.html.md This example converts the contents of a specific subdirectory within a GitHub repository into context. It uses a URL to specify the repository and the subdirectory path. ```python print(repo2ctx('https://github.com/AnswerDotAI/toolslm/tree/main/samples')) ``` -------------------------------- ### GET /symsrc Source: https://answerdotai.github.io/toolslm/inspecttools.html.md Retrieves the source code for a given symbol path. ```APIDOC ## GET /symsrc ### Description Retrieves the source code for a specified symbol path. Use '_last' to refer to the result of the previous tool call. ### Parameters #### Query Parameters - **sym** (string) - Required - Dotted symbol path (e.g., 'Interval' or 'sympy.sets.sets.Interval') or '_last'. ``` -------------------------------- ### Get Schema with String Type Annotations Source: https://answerdotai.github.io/toolslm/funccall.html Shows how `get_schema` handles functions with string-based type annotations for parameters and return types. Ensure a mandatory docstring is present. ```python def silly_test( a: 'int', # quoted type hint )->int: "Mandatory docstring" return a get_schema(silly_test) ``` -------------------------------- ### Get LLM Text File Source: https://answerdotai.github.io/toolslm/download.html Retrieves the 'llms.txt' file from a URL and expands it using `llms_txt.create_ctx()`. Useful for accessing LLM-related text data. ```python def get_llmstxt( url, optional:bool=False, n_workers:NoneType=None ): ``` ```python # print(get_llmstxt('https://llmstxt.org/llms.txt')) ``` -------------------------------- ### Get Package Path for Symbol Source: https://answerdotai.github.io/toolslm/xml.html Retrieves the root package path for a given symbol. This is useful for understanding the organizational structure of the code. ```python def sym2pkgpath( sym ): ``` ```text sym2pkgpath(test_eq) Path('/Users/jhoward/aai-ws/fastcore/fastcore') ``` -------------------------------- ### Get Package Path for Symbol Source: https://answerdotai.github.io/toolslm/xml.html.md Retrieves the root package path for a given symbol. This is useful for understanding the organizational structure of the code. ```python def sym2pkgpath( sym ): ``` ```python sym2pkgpath(test_eq) ``` -------------------------------- ### Get Package File Context Source: https://answerdotai.github.io/toolslm/inspecttools.html Retrieves XML context of all files within a symbol's top-level package. Offers extensive control over included file types, recursion, and filtering. ```python def symfiles_package( sym:str, # Dotted symbol path or "_last" for previous result types:str | list='py', # List or comma-separated str of ext types from: py, js, java, c, cpp, rb, r, ex, sh, web, doc, cfg skip_file_re:str='^_mod', # Skip files matching regex skip_folder_re:str='^(\.|__)', # Skip folders matching regex prefix:bool=False, # Include Anthropic's suggested prose intro? out:bool=True, # Include notebook cell outputs? include_base:bool=True, # Include full path in src? title:str=None, # Optional title attr for Documents element max_size:int=100000, # Skip files larger than this (bytes) max_total:int=10000000, # Max total output size in bytes readme_first:bool=False, # Prioritize README files at start of context? files_only:bool=False, # Return dict of {filename: size} instead of context? sigs_only:bool=False, # Return signatures instead of full text? (where supported by `codesigs` lib) ids:bool=True, # Include cell ids in notebooks? recursive:bool=True, # search subfolders symlinks:bool=True, # follow symlinks? file_glob:str=None, # Only include files matching glob file_re:str=None, # Only include files matching regex folder_re:str=None, # Only enter folders matching regex skip_file_glob:str=None, # Skip files matching glob ret_folders:bool=False, # return folders, not just files sort:bool=True, # sort files by name within each folder exts:str | list=None, # list or comma-separated str of exts to include ): ``` ```python # print(symfiles_package('xml')) ``` -------------------------------- ### Resolve Symbol with Indexing Source: https://answerdotai.github.io/toolslm/inspecttools.html.md Example of resolving a symbol path that includes indexing into a list-like object. ```python a = fastcore.utils.L(1) resolve('a.argfirst') ``` -------------------------------- ### Get Schema for a Simple Function Source: https://answerdotai.github.io/toolslm/funccall.html Demonstrates retrieving the schema for a simple Python function `silly_sum`. The tool use spec requires return annotations to be placed in the description. ```python s = get_schema(silly_sum) desc = s.pop('description') print(desc) s ``` -------------------------------- ### Get Package Context for Symbol Source: https://answerdotai.github.io/toolslm/xml.html.md Retrieves the contents of files within a symbol's root package. Supports various filtering and inclusion options similar to sym2folderctx. ```python def sym2pkgctx( sym, types:str | list='py', # List or comma-separated str of ext types from: py, js, java, c, cpp, rb, r, ex, sh, web, doc, cfg skip_file_re:str='^_mod', # Skip files matching regex skip_folder_re:str='^(\.|__)', # Skip folders matching regex prefix:bool=False, # Include Anthropic's suggested prose intro? out:bool=True, # Include notebook cell outputs? include_base:bool=True, # Include full path in src? title:str=None, # Optional title attr for Documents element max_size:int=100000, # Skip files larger than this (bytes) max_total:int=10000000, # Max total output size in bytes readme_first:bool=False, # Prioritize README files at start of context? files_only:bool=False, # Return dict of {filename: size} instead of context? sigs_only:bool=False, # Return signatures instead of full text? (where supported by `codesigs` lib) ids:bool=True, # Include cell ids in notebooks? recursive:bool=True, # search subfolders symlinks:bool=True, # follow symlinks? file_glob:str=None, # Only include files matching glob file_re:str=None, # Only include files matching regex folder_re:str=None, # Only enter folders matching regex skip_file_glob:str=None, # Skip files matching glob ret_folders:bool=False, # return folders, not just files sort:bool=True, # sort files by name within each folder exts:str | list=None, # list or comma-separated str of exts to include ): ``` -------------------------------- ### Get Folder File Context Source: https://answerdotai.github.io/toolslm/inspecttools.html Retrieves XML context of files within the folder where a symbol is defined. Supports various filtering and output options. ```python def symfiles_folder( sym:str, # Dotted symbol path or "_last" for previous result types:str | list='py', # List or comma-separated str of ext types from: py, js, java, c, cpp, rb, r, ex, sh, web, doc, cfg skip_file_re:str='^_mod', # Skip files matching regex prefix:bool=False, # Include Anthropic's suggested prose intro? out:bool=True, # Include notebook cell outputs? include_base:bool=True, # Include full path in src? title:str=None, # Optional title attr for Documents element max_size:int=100000, # Skip files larger than this (bytes) max_total:int=10000000, # Max total output size in bytes readme_first:bool=False, # Prioritize README files at start of context? files_only:bool=False, # Return dict of {filename: size} instead of context? sigs_only:bool=False, # Return signatures instead of full text? (where supported by `codesigs` lib) ids:bool=True, # Include cell ids in notebooks? recursive:bool=True, # search subfolders symlinks:bool=True, # follow symlinks? file_glob:str=None, # Only include files matching glob file_re:str=None, # Only include files matching regex folder_re:str=None, # Only enter folders matching regex skip_file_glob:str=None, # Skip files matching glob skip_folder_re:str=None, # Skip folders matching regex, ret_folders:bool=False, # return folders, not just files sort:bool=True, # sort files by name within each folder exts:str | list=None, # list or comma-separated str of exts to include ): ``` ```python # print(symfiles_folder('xml')) ``` -------------------------------- ### Import and Verify Module Source: https://answerdotai.github.io/toolslm/inspecttools.html.md Demonstrates importing a module and accessing its version to verify availability. ```python importmodule('fastcore.utils') fastcore.__version__ ``` -------------------------------- ### Get Symbol Type with symtype Source: https://answerdotai.github.io/toolslm/inspecttools.html.md Use `symtype` to get the type of one or more symbols. It accepts a comma-separated string of symbol paths or `"_last"` for the previous result. Returns 'SymbolNotFound' for missing symbols. ```python def symtype( syms:str, # Comma separated str list of dotted symbol paths (e.g 'Interval,a' or 'sympy.sets.sets.Interval'); "_last" for prev result ): ``` ```python symtype('a.argfirst') ``` ```python symtype('fffaa,b') ``` -------------------------------- ### Get Symbol Value Representation with symval Source: https://answerdotai.github.io/toolslm/inspecttools.html.md Use `symval` to get a list of `repr()` strings for symbol values. It accepts a comma-separated string of symbol paths or `"_last"`. Handles 'SymbolNotFound' for missing symbols. ```python def symval( syms:str, # Comma separated str list of dotted symbol paths (e.g `Interval` or `sympy.sets.sets.Interval`); "_last" for prev result ): ``` ```python a ``` ```python symval('a,foofoo') ``` -------------------------------- ### Sample Markdown for Parsing Source: https://answerdotai.github.io/toolslm/md_hier.html A sample markdown string used to demonstrate the functionality of create_heading_dict. ```python sample_md = """ # Introduction Welcome to our documentation. ## Getting Started Follow these steps to begin. ### Installation Run the following command: ```bash # Install the packackge pip install our-package ``` ### Configuration Set up your config file. ## Advanced Usage For advanced users only. # Appendix Additional resources.""" ``` -------------------------------- ### Import IPython and Textwrap Source: https://answerdotai.github.io/toolslm/inspecttools.html.md Imports necessary libraries for displaying output and formatting text. ```python from IPython.display import display,Markdown import textwrap ``` -------------------------------- ### Get Symbol Type and Value with symtype_val Source: https://answerdotai.github.io/toolslm/inspecttools.html.md Use `symtype_val` to get a list of (type, repr) tuples for symbol values. It accepts a comma-separated string of symbol paths or `"_last"`. Returns 'SymbolNotFound' for missing symbols. ```python def symtype_val( syms:str, # Comma separated str list of dotted symbol paths (e.g `Interval` or `sympy.sets.sets.Interval`); "_last" for prev result ): ``` ```python symtype_val('a,b,foofoo') ``` -------------------------------- ### GET /symtype Source: https://answerdotai.github.io/toolslm/inspecttools.html.md Retrieves the type of a symbol or list of symbols. ```APIDOC ## GET /symtype ### Description Returns the type of a symbol or a comma-separated list of symbols. Sets '_last' to the result. ### Parameters #### Query Parameters - **syms** (string) - Required - Comma-separated string list of dotted symbol paths or '_last'. ``` -------------------------------- ### GET /symtype_val Source: https://answerdotai.github.io/toolslm/inspecttools.html.md Retrieves both the type and string representation of symbol values. ```APIDOC ## GET /symtype_val ### Description Returns a list of 2-tuples containing (type, repr) for the provided symbols. ### Parameters #### Query Parameters - **syms** (string) - Required - Comma-separated string list of dotted symbol paths or '_last'. ``` -------------------------------- ### Create and print heading dictionary Source: https://answerdotai.github.io/toolslm/md_hier.html Parses the sample markdown using create_heading_dict and prints the top-level keys and the total character count of the root document's text content. ```python result = create_heading_dict(sample_md) print("Available sections:") for key in result.keys(): print(f" {key}") print(f"\nRoot document has {len(result.text)} characters of text") ``` -------------------------------- ### Initialize stateful shell Source: https://answerdotai.github.io/toolslm/shell.html Configures and returns a minimal TerminalInteractiveShell instance. ```python def get_shell()->TerminalInteractiveShell: "Get a `TerminalInteractiveShell` with minimal functionality" sh = TerminalInteractiveShell() sh.logger.log_output = sh.history_manager.enabled = False dh = sh.displayhook dh.finish_displayhook = dh.write_output_prompt = dh.start_displayhook = lambda: None dh.write_format_data = lambda format_dict, md_dict=None: None sh.logstart = sh.automagic = sh.autoindent = False sh.autocall = 0 sh.system = lambda cmd: None return sh ``` ```python shell = get_shell() ``` -------------------------------- ### GET /symval Source: https://answerdotai.github.io/toolslm/inspecttools.html.md Retrieves the string representation (repr) of symbol values. ```APIDOC ## GET /symval ### Description Returns a list of the repr() of the values for the provided symbols. ### Parameters #### Query Parameters - **syms** (string) - Required - Comma-separated string list of dotted symbol paths or '_last'. ``` -------------------------------- ### Generate schema for functions with return types and custom types Source: https://answerdotai.github.io/toolslm/funccall.html Demonstrates schema generation for functions with type hints and custom type aliases. ```python def _ret_list_test(a: int) -> list[str]: "Mandatory docstring" s = get_schema(_ret_list_test) assert 'type: array[string]' in s['description'] ``` ```python Cmd = str | list[str] def _cust_type(a: Cmd): "Mandatory docstring" s = get_schema(_cust_type) s ``` -------------------------------- ### Retrieve package context for a symbol Source: https://answerdotai.github.io/toolslm/inspecttools.html.md Returns XML context of all files within the top-level package of the provided symbol. ```python def symfiles_package( sym:str, # Dotted symbol path or "_last" for previous result types:str | list='py', # List or comma-separated str of ext types from: py, js, java, c, cpp, rb, r, ex, sh, web, doc, cfg skip_file_re:str='^_mod', # Skip files matching regex skip_folder_re:str='^(\.|__)', # Skip folders matching regex prefix:bool=False, # Include Anthropic's suggested prose intro? out:bool=True, # Include notebook cell outputs? include_base:bool=True, # Include full path in src? title:str=None, # Optional title attr for Documents element max_size:int=100000, # Skip files larger than this (bytes) max_total:int=10000000, # Max total output size in bytes readme_first:bool=False, # Prioritize README files at start of context? files_only:bool=False, # Return dict of {filename: size} instead of context? sigs_only:bool=False, # Return signatures instead of full text? (where supported by `codesigs` lib) ids:bool=True, # Include cell ids in notebooks? recursive:bool=True, # search subfolders symlinks:bool=True, # follow symlinks? file_glob:str=None, # Only include files matching glob file_re:str=None, # Only include files matching regex folder_re:str=None, # Only enter folders matching regex skip_file_glob:str=None, # Skip files matching glob ret_folders:bool=False, # return folders, not just files sort:bool=True, # sort files by name within each folder exts:str | list=None, # list or comma-separated str of exts to include ): ``` ```python # print(symfiles_package('xml')) ``` -------------------------------- ### GET /symtype Source: https://answerdotai.github.io/toolslm/inspecttools.html Returns the type of a symbol and updates the internal _last state. ```APIDOC ## GET /symtype ### Description Returns the type of a symbol, which is useful for the LLM to understand what kind of object it is dealing with. ### Parameters #### Query Parameters - **syms** (string) - Required - Comma separated string list of dotted symbol paths or '_last' for the previous result. ``` -------------------------------- ### Handle Path Types and Non-Primitive Defaults Source: https://answerdotai.github.io/toolslm/funccall.html Demonstrates handling of Path objects and using the evalable parameter to stringify non-primitive defaults. ```python def _path_test(path: Path = Path('.')): "Mandatory docstring" get_schema(_path_test) ``` ```python test_fail(lambda: ast.literal_eval(str(get_schema(_path_test))), exc=ValueError) ``` ```python def _path_test(path: Path = Path('.')): "Mandatory docstring" get_schema(_path_test, evalable=True) ``` -------------------------------- ### Get schema for python function Source: https://answerdotai.github.io/toolslm/funccall.html Retrieves the JSON schema for the python execution function. ```python get_schema(python) ``` -------------------------------- ### Find documentation URL Source: https://answerdotai.github.io/toolslm/download.html Locates the llms.txt or markdown file associated with a given URL. ```python def find_docs( url ): ``` ```python fl_url = 'https://answerdotai.github.io/fastlite' ``` ```python find_docs(fl_url) ``` ```python for o in urls: print(find_docs(o)) ``` ```python suffixes = ["/", "/tmp", "/tmp/tmp/"] for suff in suffixes: for o in urls: test_eq(find_docs(o), find_docs(o+suff)) test_eq(find_docs("https://github.com"), "https://github.com/llms.txt") test_eq(find_docs("https://github.com/AnswerDotAI"), "https://github.com/llms.txt") test_eq(find_docs("https://github.com/AnswerDotAI/"), "https://github.com/llms.txt") ``` -------------------------------- ### Initialize stateful shell Source: https://answerdotai.github.io/toolslm/shell.html.md Creates a minimal TerminalInteractiveShell instance with restricted functionality. ```python def get_shell()->TerminalInteractiveShell: "Get a `TerminalInteractiveShell` with minimal functionality" sh = TerminalInteractiveShell() sh.logger.log_output = sh.history_manager.enabled = False dh = sh.displayhook dh.finish_displayhook = dh.write_output_prompt = dh.start_displayhook = lambda: None dh.write_format_data = lambda format_dict, md_dict=None: None sh.logstart = sh.automagic = sh.autoindent = False sh.autocall = 0 sh.system = lambda cmd: None return sh ``` ```python def get_shell( )->TerminalInteractiveShell: ``` -------------------------------- ### Import necessary libraries Source: https://answerdotai.github.io/toolslm/download.html Imports the required libraries for markdown and HTML display, and testing utilities. ```python from IPython.display import Markdown,HTML from fastcore.test import * ``` -------------------------------- ### Get schema function definition Source: https://answerdotai.github.io/toolslm/funccall.html.md Signature for the get_schema function used to generate JSON schemas. ```python def get_schema( f:Union, # Function to get schema for pname:str='input_schema', # Key name for parameters evalable:bool=False, # stringify defaults that can't be literal_eval'd? skip_hidden:bool=False, # skip parameters starting with '_'? name:NoneType=None, # Override function name (useful for dotted paths like 'obj.method') )->dict: # {'name':..., 'description':..., pname:...} ``` -------------------------------- ### Create a namespace with mk_ns Source: https://answerdotai.github.io/toolslm/funccall.html.md Creates a dictionary of allowable function names to call, preventing unauthorized function execution. ```python def mk_ns( fs ): ``` ```python def sums(a, b): return a + b ns = mk_ns(sums) ns ``` ```python ns['sums'](1, 2) ``` ```python ca = ClassA() ``` -------------------------------- ### Convert Files to XML Context Source: https://answerdotai.github.io/toolslm/xml.html Use `files2ctx` to convert a list of file names into XML context. It handles notebook files and can skip files larger than a specified size. ```python def files2ctx( fnames:list, # List of file names to add to context srcs:Optional=None, # Use the labels instead of `fnames` max_size:int=None, # Skip files larger than this (bytes) out:bool=True, # Include notebook cell outputs? ids:bool=True, # Include cell ids in notebooks? nums:bool=False, # Include line numbers in notebook cell source? sigs_only:bool=False, # Only include signatures and docstrings (where supported by `codesigs` lib) prefix:bool=False, # Include Anthropic's suggested prose intro? details:Optional=None, # Optional list of dicts with additional attrs for each doc title:str=None, # Optional title attr for Documents element )->str: # XML for LM context ``` ```python fnames = ['samples/sample_core.py', 'samples/sample_styles.css'] hl_md(files2ctx(fnames, max_size=120)) ``` ```html samples/sample_core.py [Skipped: sample_core.py exceeds 120 bytes] samples/sample_styles.css .cell { margin-bottom: 1rem; } .cell > .sourceCode { margin-bottom: 0; } .cell-output > pre { margin-bottom: 0; } ``` -------------------------------- ### Get schema for a symbol with get_schema_nm Source: https://answerdotai.github.io/toolslm/funccall.html.md Generates a schema for a specific symbol in a namespace, preserving the dotted name structure. ```python def get_schema_nm( nm:str, ns, dot2dash:bool=False, kwargs:VAR_KEYWORD ): ``` ```python schema = get_schema_nm('ca.f', locals()) test_eq(schema['name'], 'ca.f') schema ``` -------------------------------- ### files2ctx - Convert files to XML context Source: https://answerdotai.github.io/toolslm/xml.html.md Converts a list of file names into an XML context, with options to control size, output inclusion, and more. Handles notebook files. ```APIDOC ## files2ctx ### Description Convert files to XML context, handling notebooks. ### Method ```python def files2ctx( fnames:list, # List of file names to add to context srcs:Optional=None, # Use the labels instead of `fnames` max_size:int=None, # Skip files larger than this (bytes) out:bool=True, # Include notebook cell outputs? ids:bool=True, # Include cell ids in notebooks? nums:bool=False, # Include line numbers in notebook cell source? sigs_only:bool=False, # Only include signatures and docstrings (where supported by `codesigs` lib) prefix:bool=False, # Include Anthropic's suggested prose intro? details:Optional=None, # Optional list of dicts with additional attrs for each doc title:str=None, # Optional title attr for Documents element )->str: # XML for LM context ``` ### Request Example ```python fnames = ['samples/sample_core.py', 'samples/sample_styles.css'] hl_md(files2ctx(fnames, max_size=120)) ``` ### Response Example ```html samples/sample_core.py [Skipped: sample_core.py exceeds 120 bytes] samples/sample_styles.css .cell { margin-bottom: 1rem; } .cell > .sourceCode { margin-bottom: 0; } .cell-output > pre { margin-bottom: 0; } ``` ``` -------------------------------- ### Exclude hidden parameters from schema Source: https://answerdotai.github.io/toolslm/funccall.html Use skip_hidden=True in get_schema to omit parameters starting with an underscore from the generated schema. ```python def test_hidden(a: int, _internal: str = "x"): "Test func" pass get_schema(test_hidden, skip_hidden=True) # should exclude _internal ``` ```python get_schema(test_hidden) ``` -------------------------------- ### GET /get_schema Source: https://answerdotai.github.io/toolslm/funccall.html Generates a JSON schema for a given class, function, or method, including parameter definitions, types, and descriptions. ```APIDOC ## GET /get_schema ### Description Generates a JSON schema for a provided Python object (class, function, or method). It maps parameters to JSON schema types and extracts docstrings for descriptions. ### Method GET ### Endpoint /get_schema ### Parameters #### Query Parameters - **f** (object) - Required - The function, class, or method to generate a schema for. - **pname** (string) - Optional - Key name for parameters (default: 'input_schema'). - **evalable** (boolean) - Optional - Whether to stringify defaults that cannot be literal_eval'd (default: False). - **skip_hidden** (boolean) - Optional - Whether to skip parameters starting with '_' (default: False). - **name** (null) - Optional - Override function name (useful for dotted paths). ### Response #### Success Response (200) - **name** (string) - The name of the function or method. - **description** (string) - The docstring or description of the object. - **input_schema** (object) - The generated JSON schema object containing properties and required fields. #### Response Example { "name": "f", "description": "object function", "input_schema": { "type": "object", "properties": { "o": {"description": "the o", "type": "object"}, "q": {"description": "", "type": "array", "prefixItems": [{"type": "integer"}, {"type": "string"}]} }, "required": ["o", "q"] } } ``` -------------------------------- ### mk_ns Source: https://answerdotai.github.io/toolslm/funccall.html.md Creates a namespace dictionary from a list of functions, mapping function names to their callable objects. ```APIDOC ## mk_ns ### Description Creates a namespace dictionary from a list of functions, mapping function names to their callable objects. ### Parameters - **fs** (list) - Required - A list of functions to include in the namespace. ### Response - **dict** - A dictionary where keys are function names and values are the function objects. ``` -------------------------------- ### sym2pkgctx Source: https://answerdotai.github.io/toolslm/xml.html Returns the contents of files in a symbol's root package. ```APIDOC ## FUNCTION sym2pkgctx ### Description Return contents of files in a symbol’s root package. ### Parameters - **sym** (any) - Required - The symbol to locate. - **types** (str|list) - Optional - List or comma-separated str of ext types. - **skip_file_re** (str) - Optional - Skip files matching regex. - **skip_folder_re** (str) - Optional - Skip folders matching regex. - **prefix** (bool) - Optional - Include Anthropic's suggested prose intro. - **out** (bool) - Optional - Include notebook cell outputs. - **include_base** (bool) - Optional - Include full path in src. - **title** (str) - Optional - Optional title attr for Documents element. - **max_size** (int) - Optional - Skip files larger than this (bytes). - **max_total** (int) - Optional - Max total output size in bytes. - **readme_first** (bool) - Optional - Prioritize README files at start of context. - **files_only** (bool) - Optional - Return dict of {filename: size} instead of context. - **sigs_only** (bool) - Optional - Return signatures instead of full text. - **ids** (bool) - Optional - Include cell ids in notebooks. - **recursive** (bool) - Optional - Search subfolders. - **symlinks** (bool) - Optional - Follow symlinks. - **file_glob** (str) - Optional - Only include files matching glob. - **file_re** (str) - Optional - Only include files matching regex. - **folder_re** (str) - Optional - Only enter folders matching regex. - **skip_file_glob** (str) - Optional - Skip files matching glob. - **ret_folders** (bool) - Optional - Return folders, not just files. - **sort** (bool) - Optional - Sort files by name within each folder. - **exts** (str|list) - Optional - List or comma-separated str of exts to include. ``` -------------------------------- ### Generate JSON Schema for List Containers Source: https://answerdotai.github.io/toolslm/funccall.html Illustrates the difference between parameterized lists and raw list types in schema generation. ```python def _list_test(l: List[int]): "Mandatory docstring" pass get_schema(_list_test) ``` ```python def _raw_list_test(l: List): "Mandatory docstring" pass get_schema(_raw_list_test) ``` -------------------------------- ### Generate XML context from files Source: https://answerdotai.github.io/toolslm/index.html.md Use folder2ctx to generate XML context from files matching a specific glob pattern. ```python print(folder2ctx('samples', prefix=False, file_glob='*.py')) ``` -------------------------------- ### Schema generation for class methods Source: https://answerdotai.github.io/toolslm/funccall.html.md Demonstrates generating schemas for class methods and callable objects. ```python class ClassA: "I am a class" def f(self, a:int): # That is `a` "Do a thing" return 1 def __call__(self, b:str): # That is `b` "Do another thing" return 2 ca = ClassA() ca.f(2) ``` ```python get_schema(ca.f) ``` ```python get_schema(ca) ``` -------------------------------- ### Import necessary libraries Source: https://answerdotai.github.io/toolslm/md_hier.html Imports the md_hier module and IPython.display.Markdown for potential use. ```python from toolslm.md_hier import * from IPython.display import Markdown ``` -------------------------------- ### Display folder2ctx help message Source: https://answerdotai.github.io/toolslm/xml.html.md Use the -h flag to display the help message for the folder2ctx command-line tool. This shows all available options for converting folder contents to XML context. ```bash !folder2ctx -h ``` -------------------------------- ### Get Schema for Nested Class Structures (Dictionary) Source: https://answerdotai.github.io/toolslm/funccall.html Shows schema generation for a class with a dictionary where values are objects. The schema uses `additionalProperties` to define the structure of dictionary values. ```python class DictConversation: "A conversation between two speakers" def __init__( self, turns:dict[str,object], # dictionary of topics and the Turns of the conversation ): store_attr() get_schema(DictConversation) ``` -------------------------------- ### Get Schema for Nested Class Structures (List) Source: https://answerdotai.github.io/toolslm/funccall.html Demonstrates schema generation for a class containing a list of custom objects. The schema includes a reference to the nested object definition. ```python class Turn: "Turn between two speakers" def __init__( self, speaker_a:str, # First speaker's message speaker_b:str, # Second speaker's message ): store_attr() class Conversation: "A conversation between two speakers" def __init__( self, turns:list[Turn], # Turns of the conversation ): store_attr() get_schema(Conversation) ``` -------------------------------- ### Retrieve folder context for a symbol Source: https://answerdotai.github.io/toolslm/inspecttools.html.md Returns XML context of files located in the folder containing the definition of the provided symbol. ```python def symfiles_folder( sym:str, # Dotted symbol path or "_last" for previous result types:str | list='py', # List or comma-separated str of ext types from: py, js, java, c, cpp, rb, r, ex, sh, web, doc, cfg skip_file_re:str='^_mod', # Skip files matching regex prefix:bool=False, # Include Anthropic's suggested prose intro? out:bool=True, # Include notebook cell outputs? include_base:bool=True, # Include full path in src? title:str=None, # Optional title attr for Documents element max_size:int=100000, # Skip files larger than this (bytes) max_total:int=10000000, # Max total output size in bytes readme_first:bool=False, # Prioritize README files at start of context? files_only:bool=False, # Return dict of {filename: size} instead of context? sigs_only:bool=False, # Return signatures instead of full text? (where supported by `codesigs` lib) ids:bool=True, # Include cell ids in notebooks? recursive:bool=True, # search subfolders symlinks:bool=True, # follow symlinks? file_glob:str=None, # Only include files matching glob file_re:str=None, # Only include files matching regex folder_re:str=None, # Only enter folders matching regex skip_file_glob:str=None, # Skip files matching glob skip_folder_re:str=None, # Skip folders matching regex, ret_folders:bool=False, # return folders, not just files sort:bool=True, # sort files by name within each folder exts:str | list=None, # list or comma-separated str of exts to include ): ``` ```python # print(symfiles_folder('xml')) ``` -------------------------------- ### Convert Files to XML Context Source: https://answerdotai.github.io/toolslm/xml.html.md Converts a list of file names into an XML string for language model context. Handles notebook files and allows filtering by file size. ```python def files2ctx( fnames:list, # List of file names to add to context srcs:Optional=None, # Use the labels instead of `fnames` max_size:int=None, # Skip files larger than this (bytes) out:bool=True, # Include notebook cell outputs? ids:bool=True, # Include cell ids in notebooks? nums:bool=False, # Include line numbers in notebook cell source? sigs_only:bool=False, # Only include signatures and docstrings (where supported by `codesigs` lib) prefix:bool=False, # Include Anthropic's suggested prose intro? details:Optional=None, # Optional list of dicts with additional attrs for each doc title:str=None, # Optional title attr for Documents element )->str: # XML for LM context ``` ```python fnames = ['samples/sample_core.py', 'samples/sample_styles.css'] hl_md(files2ctx(fnames, max_size=120)) ``` -------------------------------- ### Get Schema for Nested Class Structures (Set) Source: https://answerdotai.github.io/toolslm/funccall.html Illustrates schema generation for a class containing a set of custom objects. The schema includes `uniqueItems: True` to reflect the set's nature. ```python class SetConversation: "A conversation between two speakers" def __init__( self, turns:set[Turn], # the unique Turns of the conversation ): store_attr() get_schema(SetConversation) ``` -------------------------------- ### Get Folder Context for Symbol Source: https://answerdotai.github.io/toolslm/xml.html.md Retrieves context for a symbol's source file location. Supports filtering by file type, size, and recursion. Can include notebook outputs and cell IDs. ```python def sym2folderctx( sym, types:str | list='py', # List or comma-separated str of ext types from: py, js, java, c, cpp, rb, r, ex, sh, web, doc, cfg skip_file_re:str='^_mod', # Skip files matching regex prefix:bool=False, # Include Anthropic's suggested prose intro? out:bool=True, # Include notebook cell outputs? include_base:bool=True, # Include full path in src? title:str=None, # Optional title attr for Documents element max_size:int=100000, # Skip files larger than this (bytes) max_total:int=10000000, # Max total output size in bytes readme_first:bool=False, # Prioritize README files at start of context? files_only:bool=False, # Return dict of {filename: size} instead of context? sigs_only:bool=False, # Return signatures instead of full text? (where supported by `codesigs` lib) ids:bool=True, # Include cell ids in notebooks? recursive:bool=True, # search subfolders symlinks:bool=True, # follow symlinks? file_glob:str=None, # Only include files matching glob file_re:str=None, # Only include files matching regex folder_re:str=None, # Only enter folders matching regex skip_file_glob:str=None, # Skip files matching glob skip_folder_re:str=None, # Skip folders matching regex, ret_folders:bool=False, # return folders, not just files sort:bool=True, # sort files by name within each folder exts:str | list=None, # list or comma-separated str of exts to include ): ``` ```python # print(sym2folderctx(test_eq)) ``` -------------------------------- ### Get Schema for a Function Name Source: https://answerdotai.github.io/toolslm/funccall.html Retrieves the schema for a given symbol name within a namespace, preserving the full dotted name. This schema can describe the function's name, description, and input parameters. ```python def get_schema_nm( nm:str, ns, dot2dash:bool=False, kwargs:VAR_KEYWORD ): ``` ```python schema = get_schema_nm('ca.f', locals()) test_eq(schema['name'], 'ca.f') schema ```