### JSON Output with Source Location Example Source: https://github.com/amplify-education/python-hcl2/blob/main/docs/04_hq.md An example of JSON output when using `--json --with-location`, showing provenance and line number information. ```json { "__file__": "main.tf", "__line__": 3, "__end_line__": 7, "__column__": 1, "__end_column__": 2, "ami": "\"ami-12345\"" } ``` -------------------------------- ### Install python-hcl2 with pip Source: https://github.com/amplify-education/python-hcl2/blob/main/docs/01_getting_started.md Install the python-hcl2 library using pip. Requires Python 3.8 or higher. ```sh pip install python-hcl2 ``` -------------------------------- ### Quick Start: Query HCL Document Source: https://github.com/amplify-education/python-hcl2/blob/main/docs/02_querying.md Demonstrates basic HCL document querying using the hcl2 library. It iterates through resource blocks and extracts attribute values. ```python import hcl2 doc = hcl2.query('resource "aws_instance" "main" { ami = "abc-123" }') for block in doc.blocks("resource"): print(block.block_type, block.name_labels) ami = block.attribute("ami") if ami: print(f" ami = {ami.value}") ``` -------------------------------- ### jsontohcl2 CLI Usage Examples Source: https://github.com/amplify-education/python-hcl2/blob/main/CLAUDE.md Illustrates different ways to use the `jsontohcl2` command-line tool for converting JSON to HCL2. Covers single file conversion, diffing, semantic diffing, dry runs, fragment conversion from stdin, and formatting options. ```bash jsontohcl2 file.json # single file to stdout ``` ```bash jsontohcl2 --diff original.tf modified.json # preview text changes ``` ```bash jsontohcl2 --semantic-diff original.tf modified.json # semantic-only changes ``` ```bash jsontohcl2 --semantic-diff original.tf --diff-json m.json # semantic diff as JSON ``` ```bash jsontohcl2 --dry-run file.json # convert without writing ``` ```bash jsontohcl2 --fragment - # attribute snippets from stdin ``` ```bash jsontohcl2 --indent 4 --no-align file.json ``` -------------------------------- ### Install python-hcl2 CLI tools with pipx Source: https://github.com/amplify-education/python-hcl2/blob/main/README.md Install the CLI tools (hcl2tojson, jsontohcl2, hq) globally using pipx. This ensures the tools are available system-wide without affecting project environments. ```sh pipx install python-hcl2 ``` -------------------------------- ### hcl2tojson CLI Usage Examples Source: https://github.com/amplify-education/python-hcl2/blob/main/CLAUDE.md Demonstrates various ways to use the `hcl2tojson` command-line tool for converting HCL2 files to JSON. Includes options for single file conversion, NDJSON streaming, output directories, glob patterns, block type filtering, field projection, compact output, quiet mode, and stdin processing. ```bash hcl2tojson file.tf # single file to stdout ``` ```bash hcl2tojson --ndjson dir/ # directory → NDJSON to stdout ``` ```bash hcl2tojson a.tf b.tf -o out/ # multiple files to output dir ``` ```bash hcl2tojson --ndjson 'modules/**/*.tf' # glob + NDJSON streaming ``` ```bash hcl2tojson --only resource,module file.tf # block type filtering ``` ```bash hcl2tojson --exclude variable file.tf # exclude block types ``` ```bash hcl2tojson --fields cpu,memory file.tf # field projection ``` ```bash hcl2tojson --compact file.tf # single-line JSON ``` ```bash hcl2tojson -q dir/ -o out/ # quiet (no stderr progress) ``` ```bash echo 'x = 1' | hcl2tojson # stdin (no args needed) ``` -------------------------------- ### List All Resource Names Source: https://github.com/amplify-education/python-hcl2/blob/main/docs/05_hq_examples.md Lists the name labels of all resources found in the specified directory. Use `--value --no-filename` to get a clean list of names. ```sh hq 'resource[*] | .name_labels' infra/ --value --no-filename ``` -------------------------------- ### Install python-hcl2 with pip Source: https://github.com/amplify-education/python-hcl2/blob/main/README.md Install the python-hcl2 package using pip. This command installs the library for use within your Python projects. ```sh pip3 install python-hcl2 ``` -------------------------------- ### Get attribute names Source: https://github.com/amplify-education/python-hcl2/blob/main/docs/04_hq.md Query for the '.name' property on attribute views to list all attribute names. ```sh # Get attribute names hq '*[*] | .name' main.tf --value ``` -------------------------------- ### Control HCL2 Output Formatting with DeserializerOptions and FormatterOptions Source: https://context7.com/amplify-education/python-hcl2/llms.txt Customize HCL2 output using `DeserializerOptions` for parsing behavior and `FormatterOptions` for whitespace and alignment. This example shows how to configure heredocs, object element separators, and indentation. ```python from hcl2 import dumps, DeserializerOptions, FormatterOptions data = { "__is_block__": True, "resource": [{"aws_s3_bucket": {"logs": { "__is_block__": True, "bucket": '"my-logs-bucket"', "description": '"Stores access logs"', }}}] } text = dumps( data, deserializer_options=DeserializerOptions( heredocs_to_strings=False, # keep heredocs strings_to_heredocs=False, # don't convert \n strings to heredocs object_elements_colon=False, # use = (not :) in object elements object_elements_trailing_comma=True # trailing commas in objects ), formatter_options=FormatterOptions( indent_length=2, # 2-space indent open_empty_blocks=True, # expand empty blocks vertically_align_attributes=True, # align = signs vertically_align_object_elements=True, # align = in objects ), ) print(text) # resource "aws_s3_bucket" "logs" { # bucket = "my-logs-bucket" # description = "Stores access logs" # } ``` -------------------------------- ### Query HCL2 files with hq Source: https://github.com/amplify-education/python-hcl2/blob/main/README.md Employ the hq CLI tool for structural queries on HCL2 files. Examples show querying specific attributes and retrieving all variables as JSON. ```sh hq 'resource.aws_instance.main.ami' main.tf hq 'variable[*]' variables.tf --json ``` -------------------------------- ### Get function call names Source: https://github.com/amplify-education/python-hcl2/blob/main/docs/04_hq.md Use a path with a wildcard and a type filter to select function calls and retrieve their '.name' property. ```sh # Get function call names hq '*..function_call:*[*] | .name' main.tf --value ``` -------------------------------- ### JSON Error Output Example Source: https://github.com/amplify-education/python-hcl2/blob/main/docs/04_hq.md When using flags like --json, --describe, or --schema, errors are outputted as JSON objects to stderr. ```json {"error": "query_syntax", "message": "Invalid path segment: '123' in '123invalid'", "query": "123invalid"} ``` ```json {"error": "unsafe_expression", "message": "comprehensions are not allowed", "expression": "[x for x in _]"} ``` ```json {"error": "parse_error", "message": "Unexpected token ..."} ``` -------------------------------- ### Get keys of an object using the 'keys' builtin Source: https://github.com/amplify-education/python-hcl2/blob/main/docs/04_hq.md Use the 'keys' builtin function within a pipe to list all keys of an object. ```sh # Builtins hq 'x | keys' file.tf --json ``` -------------------------------- ### Programmatically construct HCL2 documents with hcl2.Builder Source: https://context7.com/amplify-education/python-hcl2/llms.txt Builder creates dicts with the correct __is_block__ markers so dumps can distinguish HCL blocks from plain objects. Use block() to add blocks (returns the child Builder for chaining), then call build() to get the final dict. ```python import hcl2 doc = hcl2.Builder() # Top-level resource block with labels res = doc.block("resource", labels=["aws_instance", "web"], ami='"abc-123"', instance_type='"t2.micro"') # Nested block inside the resource res.block("tags", Name='"HelloWorld"', Env='"prod"') ``` -------------------------------- ### Combine skip labels with wildcards and pipe to get block types Source: https://github.com/amplify-education/python-hcl2/blob/main/docs/04_hq.md This query retrieves the block type for all top-level blocks, skipping labels and piping the results. ```sh # Combine with wildcards hq '*~[*] | .block_type' main.tf --value ``` -------------------------------- ### Query HCL2 Document with DocumentView Source: https://context7.com/amplify-education/python-hcl2/llms.txt Use `hcl2.query` to get a `DocumentView` for navigating HCL2 structure without full serialization. Supports iterating over blocks and accessing attributes. Requires the `hcl2` library. ```python import hcl2 from hcl2.query.body import DocumentView text = ''' resource "aws_instance" "main" { ami = "ami-12345" instance_type = "t3.micro" tags = { Name = "prod-web" } } variable "region" { default = "us-east-1" } ''' doc = hcl2.query(text) # Iterate over all resource blocks for block in doc.blocks("resource"): print(block.block_type) print(block.labels) print(block.name_labels) ami = block.attribute("ami") if ami: print(ami.value) print(ami.to_hcl()) # Single attribute lookup var_block = doc.blocks("variable", "region")[0] default_attr = var_block.attribute("default") print(default_attr.value) # From a file doc = DocumentView.parse_file("main.tf") ``` -------------------------------- ### Get all labels including block type Source: https://github.com/amplify-education/python-hcl2/blob/main/docs/04_hq.md Access the '.labels' property to get a list of all labels associated with a block, including its type. ```sh # Get all labels including block type hq 'resource[*] | .labels' main.tf --json ``` -------------------------------- ### Run Unit Tests Source: https://github.com/amplify-education/python-hcl2/blob/main/CLAUDE.md Execute all unit tests using the unittest discover command. Tests are located in the 'test' directory and follow the 'test_*.py' naming convention. ```bash python -m unittest discover -s test -p "test_*.py" -v ``` -------------------------------- ### Enable Metadata Keys in v8 Source: https://github.com/amplify-education/python-hcl2/blob/main/docs/06_migrating_to_v8.md The v7 metadata keys `__start_line__` and `__end_line__` are opt-in in v8 via `with_meta=True`. ```python opts = SerializationOptions(with_meta=True) ``` -------------------------------- ### Get all variable blocks Source: https://github.com/amplify-education/python-hcl2/blob/main/docs/04_hq.md Use this command to retrieve all blocks of type 'variable'. ```sh # Get all variable blocks hq 'variable[*]' variables.tf ``` -------------------------------- ### Parse HCL from File Source: https://github.com/amplify-education/python-hcl2/blob/main/docs/02_querying.md Shows how to parse an HCL configuration directly from a file using DocumentView. ```python from hcl2.query import DocumentView doc = DocumentView.parse_file("main.tf") ``` -------------------------------- ### Get name labels of resource blocks Source: https://github.com/amplify-education/python-hcl2/blob/main/docs/04_hq.md Retrieve the '.name_labels' property for resource blocks, which includes labels after the block type. ```sh # Get name labels (labels after the block type) hq 'resource[*] | .name_labels' main.tf --json ``` -------------------------------- ### Select modules by source string Source: https://context7.com/amplify-education/python-hcl2/llms.txt Filters modules based on whether their 'source' attribute contains the substring 'docker'. ```bash hq 'module~[select(.source | contains("docker"))]' dir/ --value ``` -------------------------------- ### Get block types using property accessors Source: https://github.com/amplify-education/python-hcl2/blob/main/docs/04_hq.md Access the '.block_type' property on BlockViews to retrieve the type of each resource block. ```sh # Get block types hq 'resource[*] | .block_type' main.tf --value ``` -------------------------------- ### Dump Full View API Schema with --schema Source: https://github.com/amplify-education/python-hcl2/blob/main/docs/04_hq.md Use the --schema flag to dump the complete view API hierarchy as JSON. This command does not require a query or file. ```sh hq --schema ``` -------------------------------- ### ObjectView: Accessing Object Keys and Values Source: https://github.com/amplify-education/python-hcl2/blob/main/docs/02_querying.md Demonstrates how to use ObjectView to access keys, values, and entries within HCL objects (maps). ```python from hcl2.query.containers import ObjectView from hcl2.rules.containers import ObjectRule node = find_first(doc.attribute("tags").raw, ObjectRule) ov = ObjectView(node) ov.keys # ["Name", "Env"] ov.get("Name") # NodeView for the value ov.entries # List[Tuple[str, NodeView]] ``` -------------------------------- ### Output raw values only Source: https://github.com/amplify-education/python-hcl2/blob/main/docs/04_hq.md Use the --value flag to get only the raw values of the queried attributes, without JSON formatting. ```sh # Output raw values only hq 'resource.aws_instance.main.ami' main.tf --value ``` -------------------------------- ### Query all resource blocks regardless of labels Source: https://github.com/amplify-education/python-hcl2/blob/main/docs/04_hq.md Use 'resource~[*]' to select all resource blocks, ignoring their specific labels. ```sh # All resource blocks regardless of labels hq 'resource~[*]' main.tf ``` -------------------------------- ### Get the length of a value using the 'length' builtin Source: https://github.com/amplify-education/python-hcl2/blob/main/docs/04_hq.md Use the 'length' builtin function to determine the size of a value (e.g., array or string length). ```sh hq 'x | length' file.tf --value ``` -------------------------------- ### HCL2 v8 Serialization Options for v7 Compatibility Source: https://github.com/amplify-education/python-hcl2/blob/main/docs/06_migrating_to_v8.md Demonstrates how to configure SerializationOptions to mimic v7 behavior, specifically for loading data. This approach restores the v7 dict shape but disables round-trip support and comment preservation. ```python import hcl2 from hcl2 import SerializationOptions V7_COMPAT = SerializationOptions( strip_string_quotes=True, explicit_blocks=False, with_comments=False, ) data = hcl2.load(f, serialization_options=V7_COMPAT) ``` -------------------------------- ### Query Terraform Files with HQ Source: https://github.com/amplify-education/python-hcl2/blob/main/docs/05_hq_examples.md Demonstrates basic hq usage for querying Terraform files, directories, multiple files, and using glob patterns. Includes options for JSON or ndjson output and showing line numbers. ```sh hq 'resource[*]' main.tf --json # Query a file hq 'resource[*]' infra/ --ndjson # Query a directory (recursive) hq 'resource[*]' main.tf vars.tf --json # Multiple files hq 'resource[*]' 'modules/**/*.tf' --json # Glob pattern hq 'resource[*]' infra/ --json --with-location # With line numbers hq 'locals.app_name?' dir/ --value # Exit 0 even if missing ``` -------------------------------- ### Configure Serialization Options in Python HCL2 Source: https://context7.com/amplify-education/python-hcl2/llms.txt Demonstrates how to configure serialization options for the `loads` function to control the output format, including comments, metadata, and string quoting. Use `SerializationOptions` to fine-tune the parsing and output. ```python opts = SerializationOptions( with_comments=True, # include __comments__ / __inline_comments__ keys with_meta=False, # omit __start_line__ / __end_line__ wrap_objects=False, # keep nested objects as Python dicts wrap_tuples=False, # keep tuples as Python lists explicit_blocks=True, # keep __is_block__ markers (required for round-trip) preserve_heredocs=True, # keep < dict # v8 rule_tree = hcl2.transform(lark_tree, discard_comments=False) # -> StartRule data = hcl2.serialize(rule_tree, serialization_options=opts) # -> dict ``` -------------------------------- ### Build HCL2 Document from Doc Object Source: https://context7.com/amplify-education/python-hcl2/llms.txt Demonstrates building an HCL2 string from a structured document object. Requires the `hcl2` library and a `doc` object configured with blocks and attributes. ```python import hcl2 doc = hcl2.Document() doc.block("provider", labels=["aws"], region='"us-east-1"') doc.block("variable", labels=["instance_type"], description='"EC2 instance type"', default='"t2.micro"') hcl_string = hcl2.dumps(doc.build()) print(hcl_string) ``` -------------------------------- ### Query resources skipping labels Source: https://context7.com/amplify-education/python-hcl2/llms.txt Selects all resource bodies, skipping any labels that match the '~' pattern. ```bash hq 'resource~[*]' main.tf --json ``` -------------------------------- ### Update load()/loads() Signature in v8 Source: https://github.com/amplify-education/python-hcl2/blob/main/docs/06_migrating_to_v8.md The `with_meta` parameter is replaced by `serialization_options`. All parameters are now keyword-only. ```python # v7 data = hcl2.load(f, with_meta=True) data = hcl2.loads(text, with_meta=True) # v8 from hcl2 import SerializationOptions data = hcl2.load(f, serialization_options=SerializationOptions(with_meta=True)) data = hcl2.loads(text, serialization_options=SerializationOptions(with_meta=True)) ``` -------------------------------- ### Input: Multiple Files and Globs with hq Source: https://github.com/amplify-education/python-hcl2/blob/main/docs/04_hq.md hq accepts multiple file arguments, directories, and glob patterns for querying. Stdin is used by default when no file is provided. ```sh # Multiple files hq 'resource[*]' file1.tf file2.tf --json ``` ```sh # Directory (walks for .tf, .hcl, .tfvars) hq 'variable[*]' modules/ ``` ```sh # Mix files and directories hq 'resource[*]' main.tf modules/ --json ``` ```sh # Glob patterns (expanded by hq, not the shell) hq 'resource[*]' 'modules/**/*.tf' --json ``` ```sh # Stdin (default when no FILE given) echo 'x = 1' | hq 'x' --value ``` -------------------------------- ### Select Modules by Source with hq Source: https://github.com/amplify-education/python-hcl2/blob/main/docs/04_hq.md Filter and select modules that source 'docker' using hq's select and string functions. ```sh # Select + string functions — modules sourcing "docker" hq 'module~[select(.source | contains("docker"))]' dir/ --value ``` -------------------------------- ### Accessing Block Information Source: https://github.com/amplify-education/python-hcl2/blob/main/docs/02_querying.md Demonstrates how to access properties of a BlockView object, such as its type, labels, and body. ```python block = doc.blocks("resource", "aws_instance")[0] block.block_type # "resource" block.labels # ["resource", "aws_instance", "main"] block.name_labels # ["aws_instance", "main"] block.body # BodyView ``` -------------------------------- ### Source Location (`--with-location`) Source: https://github.com/amplify-education/python-hcl2/blob/main/docs/04_hq.md Add `--with-location` to `--json` or `--ndjson` output to include source file and line/column numbers for each result. ```sh hq 'resource[*]' main.tf --json --with-location ``` -------------------------------- ### Index query for the first variable Source: https://github.com/amplify-education/python-hcl2/blob/main/docs/04_hq.md Use array indexing like '[0]' to select a specific item from a list of blocks. ```sh # Index: first variable only hq 'variable[0]' variables.tf ``` -------------------------------- ### Build HCL2 structure programmatically Source: https://github.com/amplify-education/python-hcl2/blob/main/README.md Utilize the hcl2.Builder to construct HCL2 documents from scratch in Python. This allows for dynamic generation of HCL2 configurations by adding blocks and attributes. ```python import hcl2 doc = hcl2.Builder() res = doc.block("resource", labels=["aws_instance", "web"], ami="abc-123", instance_type="t2.micro") res.block("tags", Name="HelloWorld") hcl_string = hcl2.dumps(doc.build()) ``` -------------------------------- ### DocumentView Parsing Methods Source: https://github.com/amplify-education/python-hcl2/blob/main/docs/02_querying.md Illustrates different ways to parse HCL content into a DocumentView object, including from strings, files, and file objects. ```python doc = DocumentView.parse(text) # from string doc = DocumentView.parse_file("main.tf") # from file doc = hcl2.query(text) # convenience alias doc = hcl2.query(open("main.tf")) # also accepts file objects ``` -------------------------------- ### TupleView: Accessing Tuple Elements Source: https://github.com/amplify-education/python-hcl2/blob/main/docs/02_querying.md Illustrates how to work with TupleView objects to access elements within HCL tuples (lists). ```python from hcl2.query.containers import TupleView from hcl2.walk import find_first from hcl2.rules.containers import TupleRule doc = DocumentView.parse('x = [1, 2, 3]\n') node = find_first(doc.attribute("x").raw, TupleRule) tv = TupleView(node) len(tv) # 3 tv[0] # NodeView for the first element tv.elements # List[NodeView] ``` -------------------------------- ### Preview JSON to HCL2 diff Source: https://context7.com/amplify-education/python-hcl2/llms.txt Compares a modified JSON file against an original HCL2 file and shows the changes as a unified diff. ```bash jsontohcl2 --diff original.tf modified.json ``` -------------------------------- ### Object Construction with Multiple Fields using hq Source: https://github.com/amplify-education/python-hcl2/blob/main/docs/04_hq.md Construct objects with multiple fields, such as type and name, for each resource using hq. ```sh # Object construction — multiple fields per result hq 'resource[*] | {type: .block_type, name: .name_labels}' main.tf --json ``` -------------------------------- ### ForTupleView: Inspecting For-Tuple Expressions Source: https://github.com/amplify-education/python-hcl2/blob/main/docs/02_querying.md Shows how to inspect the components of a for-tuple expression in HCL, including iterators, iterable, and value expressions. ```python from hcl2.query.for_exprs import ForTupleView from hcl2.rules.for_expressions import ForTupleExprRule doc = DocumentView.parse('x = [for item in var.list : item]\n') node = find_first(doc.raw, ForTupleExprRule) fv = ForTupleView(node) fv.iterator_name # "item" fv.second_iterator_name # None (or "v" for "k, v in ...") fv.iterable # NodeView fv.value_expr # NodeView fv.has_condition # bool fv.condition # NodeView | None ``` -------------------------------- ### FunctionCallView: Inspecting Function Calls Source: https://github.com/amplify-education/python-hcl2/blob/main/docs/02_querying.md Shows how to inspect HCL function calls, including the function name and its arguments. ```python from hcl2.query.functions import FunctionCallView from hcl2.rules.functions import FunctionCallRule doc = DocumentView.parse('x = length(var.list)\n') node = find_first(doc.raw, FunctionCallRule) fv = FunctionCallView(node) fv.name # "length" fv.args # List[NodeView] fv.has_ellipsis # bool ``` -------------------------------- ### HCL2 v8 Pipeline Stages Source: https://github.com/amplify-education/python-hcl2/blob/main/docs/06_migrating_to_v8.md Illustrates the forward and reverse data transformation pipeline in HCL2 v8. The forward path converts HCL text to a dictionary, while the reverse path converts a dictionary back to HCL text. ```text Forward: HCL text -> parses_to_tree() -> transform() -> serialize() -> dict Reverse: dict -> from_dict() -> reconstruct() -> HCL text ``` -------------------------------- ### Control Serialization Output with SerializationOptions Source: https://context7.com/amplify-education/python-hcl2/llms.txt Use `SerializationOptions` with `hcl2.loads` or `hcl2.serialize` to control the output dictionary's shape. Defaults ensure lossless round-trips. Requires the `hcl2` library. ```python from hcl2 import loads, SerializationOptions text = '# Configure provider\nregion = "us-east-1"\n' # Example usage (assuming loads is used, serialize would take similar options) data = loads(text, serialization_options=SerializationOptions(strip_trailing_newline=True)) print(data) ``` -------------------------------- ### Resource Inventory with Tags Source: https://github.com/amplify-education/python-hcl2/blob/main/docs/05_hq_examples.md Generates a resource inventory including the resource type and its associated tags, formatted as ndjson. ```sh hq 'resource~[*] | {type: .name_labels, tags}' infra/prod/ --ndjson ``` -------------------------------- ### Query IAM and Networking Resources Source: https://github.com/amplify-education/python-hcl2/blob/main/docs/05_hq_examples.md Demonstrates querying for specific IAM and networking resource types, such as IAM policy documents, IAM roles, and Route 53 records. ```sh hq 'data.aws_iam_policy_document[*] | .name_labels' infra/ --value hq 'resource.aws_iam_role[*] | .name_labels' infra/ --value hq 'resource.aws_route53_record[*] | .name_labels' infra/ --value ``` -------------------------------- ### Programmatic HCL Document Construction with Builder Source: https://github.com/amplify-education/python-hcl2/blob/main/docs/03_advanced_api.md The `Builder` class facilitates programmatic creation of HCL documents, correctly marking blocks and attributes. Use `dumps` to serialize the built document to an HCL string. ```python import hcl2 doc = hcl2.Builder() res = doc.block("resource", labels=["aws_instance", "web"], ami="abc-123", instance_type="t2.micro") res.block("tags", Name="HelloWorld") hcl_string = hcl2.dumps(doc.build()) ``` -------------------------------- ### List Module Sources Source: https://github.com/amplify-education/python-hcl2/blob/main/docs/05_hq_examples.md Retrieves the source attribute for all modules. The `~` operator skips module names and `| .source | .value` unwraps the source string. ```sh hq 'module~[*] | .source | .value' infra/ --value ``` -------------------------------- ### Raw Lark Pipeline for HCL2 Parsing Source: https://context7.com/amplify-education/python-hcl2/llms.txt Use `hcl2.parses_to_tree` for the raw `lark.Tree` and `hcl2.transform` to convert it to a typed `StartRule`. This allows hooking into the lowest parse pipeline levels. Requires the `hcl2` library. ```python import hcl2 text = 'resource "aws_instance" "web" { ami = "abc-123" }\n' # Step 1: raw Lark parse tree lark_tree = hcl2.parses_to_tree(text) # Step 2: typed LarkElement tree rule_tree = hcl2.transform(lark_tree) # Step 3: Python dict data = hcl2.serialize(rule_tree) # Full pipeline equivalent to hcl2.loads(text): assert data == hcl2.loads(text) ``` -------------------------------- ### Any and All for List Element Checks Source: https://github.com/amplify-education/python-hcl2/blob/main/docs/04_hq.md Use 'any' and 'all' to iterate over list-valued accessors and test predicates on elements. This allows for conditions on list contents. ```sh hq '*..tuple:*[*] | select(any(.elements; .type == "function_call"))' file.tf ``` ```sh hq '*..tuple:*[*] | select(all(.elements; .type == "node"))' file.tf ``` ```sh hq '*..tuple:*[*] | select(any(.elements; .type == "function_call" or .type == "tuple"))' file.tf ``` -------------------------------- ### List Specific Resource Type Names Source: https://github.com/amplify-education/python-hcl2/blob/main/docs/05_hq_examples.md Filters and lists the name labels for a specific resource type, such as 'aws_s3_bucket'. ```sh hq 'resource.aws_s3_bucket[*] | .name_labels' infra/ --value ``` -------------------------------- ### Find Resources Using for_each or count Source: https://github.com/amplify-education/python-hcl2/blob/main/docs/05_hq_examples.md Lists resources that utilize either the `for_each` meta-argument or the `count` meta-argument. ```sh hq 'resource~[select(.for_each)] | .name_labels' infra/ --value hq 'resource~[select(.count)] | .count | .value' infra/ --value ``` -------------------------------- ### Traverse HCL Tree with walk Functions Source: https://github.com/amplify-education/python-hcl2/blob/main/docs/02_querying.md Utilize functions from the hcl2.walk module to traverse the parsed HCL Abstract Syntax Tree (AST). Imports are required for walk, walk_rules, walk_semantic, find_all, find_first, and ancestors. ```python from hcl2.walk import walk, walk_rules, walk_semantic, find_all, find_first, ancestors from hcl2.rules.base import AttributeRule tree = hcl2.parses('x = 1 y = 2 ') # All nodes depth-first (including tokens) for node in walk(tree): print(node) # Only LarkRule nodes for rule in walk_rules(tree): print(rule) # Only semantic rules (skip NewLineOrCommentRule) for rule in walk_semantic(tree): print(rule) # Find specific rule types attrs = list(find_all(tree, AttributeRule)) first_attr = find_first(tree, AttributeRule) # Walk up the parent chain for parent in ancestors(first_attr): print(parent) ``` -------------------------------- ### JSON Provenance (`__file__`) Source: https://github.com/amplify-education/python-hcl2/blob/main/docs/04_hq.md When querying multiple files with `--json` or `--ndjson`, results automatically include a `"__file__"` key. Use `--no-filename` to suppress. ```sh hq 'resource[*]' main.tf --json ``` -------------------------------- ### Include line metadata in JSON Source: https://context7.com/amplify-education/python-hcl2/llms.txt Includes source code line number and file information in the JSON output. ```bash hcl2tojson --with-meta main.tf ``` -------------------------------- ### Run tests with tox Source: https://github.com/amplify-education/python-hcl2/blob/main/README.md Execute linters and unit tests using tox. You can run all tests or specify a particular environment, such as 'py310-unit' for Python 3.10 unit tests. ```sh tox tox -e py310-unit tox -l ``` -------------------------------- ### Output Modes: JSON and Value with hq Source: https://github.com/amplify-education/python-hcl2/blob/main/docs/04_hq.md Demonstrates the `--json` and `--value` output modes. `--json` reconstructs HCL to JSON, while `--value` provides raw values, auto-unwrapping single-key dicts. ```sh hq 'resource[*]' dir/ --json ``` ```sh hq 'variable[*]' file.tf --value ``` -------------------------------- ### Piping hq to jq for String Transformation Source: https://github.com/amplify-education/python-hcl2/blob/main/docs/04_hq.md Transform module source strings to lowercase using hq for module sources and jq for mapping. ```sh # Mapping — transform each result hq 'module~[*] | .source' dir/ --ndjson | jq -r 'ascii_downcase' ``` -------------------------------- ### Builtin Transforms: Keys, Values, Length Source: https://github.com/amplify-education/python-hcl2/blob/main/docs/04_hq.md Utilize builtin transforms like 'keys', 'values', and 'length' as terminal pipe stages. These provide summary information about objects and collections. ```sh hq 'tags | keys' file.tf --json # one JSON array ``` ```sh hq 'tags | keys[*]' file.tf --value # one key per line ``` ```sh hq 'items | length' file.tf --value ``` -------------------------------- ### Parallel Processing with hq Source: https://github.com/amplify-education/python-hcl2/blob/main/docs/04_hq.md hq automatically uses multiprocessing for 20+ files with `--json` or `--ndjson`. Control this with `--jobs`. ```sh # Auto-parallel (default for 20+ files with --json/--ndjson) hq 'resource[*]' large-monorepo/ --ndjson ``` ```sh # Force serial processing hq 'resource[*]' large-monorepo/ --ndjson --jobs 0 ``` ```sh # Explicit worker count hq 'resource[*]' large-monorepo/ --ndjson --jobs 8 ``` -------------------------------- ### Comments (`--with-comments`) Source: https://github.com/amplify-education/python-hcl2/blob/main/docs/04_hq.md Include comments in the serialized output by adding `--with-comments` to `--json` or `--ndjson` commands. ```sh hq 'resource[*]' main.tf --json --with-comments ``` -------------------------------- ### Find Resources Without Tags Source: https://github.com/amplify-education/python-hcl2/blob/main/docs/05_hq_examples.md Identifies resources that do not have a 'tags' attribute defined, which can indicate a compliance violation. ```sh hq 'resource~[select(not .tags)] | .name_labels' infra/ --value ``` -------------------------------- ### Piping hq to jq for Default Value Filling Source: https://github.com/amplify-education/python-hcl2/blob/main/docs/04_hq.md Use hq to extract resource names and tags, then pipe to jq to fill missing tag values with 'none'. ```sh # Defaults — fill missing values hq 'resource~[*] | {name: .name_labels, tags}' main.tf --json | jq '.tags // "none"' ``` -------------------------------- ### Compare HCL Files Structurally Source: https://github.com/amplify-education/python-hcl2/blob/main/docs/04_hq.md Use the --diff flag to compare two HCL files. Output can be in standard format or JSON. ```sh hq file1.tf --diff file2.tf ``` ```sh hq file1.tf --diff file2.tf --json ``` -------------------------------- ### Piping NDJSON to jq Source: https://github.com/amplify-education/python-hcl2/blob/main/docs/04_hq.md Demonstrates piping the `--ndjson` output of hq to `jq` for further JSON processing, such as extracting specific fields. ```sh hq 'resource[*]' dir/ --ndjson | jq '.tags' ``` -------------------------------- ### hcl2.Builder Source: https://context7.com/amplify-education/python-hcl2/llms.txt Provides a programmatic way to construct HCL2 documents using a builder pattern. The `Builder` class helps create dictionaries with the necessary markers for correct serialization by `dumps`. ```APIDOC ## `hcl2.Builder` — Programmatically construct HCL2 documents `Builder` creates dicts with the correct `__is_block__` markers so `dumps` can distinguish HCL blocks from plain objects. Use `block()` to add blocks (returns the child `Builder` for chaining), then call `build()` to get the final dict. ### Methods - `block(name, *labels, **attributes)`: Adds a new HCL block. Returns a `Builder` instance for the nested content. - `build()`: Returns the constructed HCL2 document as a Python dictionary. ### Request Example ```python import hcl2 doc = hcl2.Builder() # Top-level resource block with labels res = doc.block("resource", labels=["aws_instance", "web"], ami='"abc-123"', instance_type='"t2.micro"') # Nested block inside the resource res.block("tags", Name='"HelloWorld"', Env='"prod"') # Get the final dictionary representation hcl_dict = doc.build() ``` ``` -------------------------------- ### Find Modules with Auto-Deploy Attribute Source: https://github.com/amplify-education/python-hcl2/blob/main/docs/05_hq_examples.md Identifies modules that have an 'auto_deploy' attribute set and extracts its value. ```sh hq 'module~[select(.auto_deploy)] | .auto_deploy | .value' infra/ --value ``` -------------------------------- ### Custom JSON to HCL2 formatting Source: https://context7.com/amplify-education/python-hcl2/llms.txt Applies custom indentation and alignment settings during JSON to HCL2 conversion. ```bash jsontohcl2 --indent 4 --no-align output.json ``` -------------------------------- ### Select resources by AMI regex Source: https://context7.com/amplify-education/python-hcl2/llms.txt Filters AWS instances based on whether their 'ami' attribute matches a specific regex pattern. ```bash hq 'resource.aws_instance~[select(.ami | test("^ami-[0-9]+"))]' dir/ --value ``` -------------------------------- ### Navigate into a specific resource Source: https://github.com/amplify-education/python-hcl2/blob/main/docs/04_hq.md Access a specific attribute within a nested resource block. ```sh # Navigate into a specific resource hq 'resource.aws_instance.main.ami' main.tf ``` -------------------------------- ### Output query results as JSON Source: https://github.com/amplify-education/python-hcl2/blob/main/docs/04_hq.md Append the --json flag to format the output of your query as a JSON array. ```sh # Output as JSON hq 'variable[*]' variables.tf --json ``` -------------------------------- ### Output Modes: Raw and NDJSON with hq Source: https://github.com/amplify-education/python-hcl2/blob/main/docs/04_hq.md Illustrates `--raw` mode for stripping quotes from strings and `--ndjson` for newline-delimited JSON output, suitable for streaming. ```sh # Raw output hq 'resource[*]' dir/ --raw ``` ```sh # NDJSON output hq 'resource[*]' dir/ --ndjson ``` -------------------------------- ### Hybrid query mode (structural + Python) Source: https://context7.com/amplify-education/python-hcl2/llms.txt Combines structural HCL2 path querying with Python expressions to filter and select data. ```bash hq 'variable[select(.default)]::name_labels[0]' variables.tf --value ``` -------------------------------- ### ConditionalView: Inspecting Ternary Expressions Source: https://github.com/amplify-education/python-hcl2/blob/main/docs/02_querying.md Demonstrates how to access the condition, true value, and false value of a ternary conditional expression in HCL. ```python from hcl2.query.expressions import ConditionalView from hcl2.rules.expressions import ConditionalRule doc = DocumentView.parse('x = var.enabled ? "on" : "off"\n') node = find_first(doc.raw, ConditionalRule) cv = ConditionalView(node) cv.condition # NodeView over the condition expression cv.true_val # NodeView over the true branch cv.false_val # NodeView over the false branch ``` -------------------------------- ### Load HCL with Comments Source: https://github.com/amplify-education/python-hcl2/blob/main/docs/01_getting_started.md Enable `with_comments=True` in `SerializationOptions` to include comments as `__comments__` and `__inline_comments__` keys in the parsed dictionary. Note that comments are read-only and not restored during dumping. ```python from hcl2 import loads, SerializationOptions data = loads( "# Configure the provider\nx = 1\n", serialization_options=SerializationOptions(with_comments=True), ) data["__comments__"] # [{"value": "Configure the provider"}] ``` -------------------------------- ### Piping hq to jq for Aggregation Source: https://github.com/amplify-education/python-hcl2/blob/main/docs/04_hq.md Group resources by their block type using hq for resource extraction and jq for aggregation. ```sh # Aggregation — group or sort across results hq 'resource[*]' dir/ --ndjson | jq -s 'group_by(.block_type)' ``` -------------------------------- ### Basic Label Traversal Source: https://github.com/amplify-education/python-hcl2/blob/main/docs/04_hq.md Access a specific attribute within a resource by traversing labels. This is a straightforward way to pinpoint data. ```sh hq 'resource.aws_instance.main | .ami' main.tf ``` -------------------------------- ### Hybrid Queries with Python Evaluation Source: https://github.com/amplify-education/python-hcl2/blob/main/docs/04_hq.md Combine structural queries with Python expressions using '::'. The Python expression runs for each result from the structural path, with '_' bound to the result. ```sh hq 'variable[*]::name_labels' variables.tf ``` ```sh hq 'variable[*]::block_type' variables.tf --value ``` ```sh hq 'resource.aws_instance[*].tags::entries()' main.tf ``` ```sh hq 'variable[*]::len(_.name_labels)' variables.tf --value ``` -------------------------------- ### Serialize LarkElement Tree to Python Dict Source: https://github.com/amplify-education/python-hcl2/blob/main/docs/03_advanced_api.md Use `serialize` to convert a LarkElement tree into a Python dictionary. Options like `with_meta=True` can be passed via `SerializationOptions` for more detailed output. ```python tree = hcl2.parses('x = 1') data = hcl2.serialize(tree) # or with options: from hcl2 import SerializationOptions data = hcl2.serialize(tree, serialization_options=SerializationOptions(with_meta=True)) ``` -------------------------------- ### PostLexer Class Structure Source: https://github.com/amplify-education/python-hcl2/blob/main/CLAUDE.md Shows the basic structure of the `PostLexer` class, designed to transform token streams between the lexer and parser. It outlines how to add new transformation passes by creating private methods and chaining them in the `process` method. ```python class PostLexer: def process(self, stream): # ... chain passes here ... yield from self._merge_newlines_into_operators(stream) ``` -------------------------------- ### HCL2 Parsing Pipeline Source: https://github.com/amplify-education/python-hcl2/blob/main/CLAUDE.md Illustrates the different processing pipelines for HCL2 text, from lexing to Python dictionary or direct tree reconstruction. The 'Direct' pipeline preserves whitespace and comments. ```text Forward: HCL2 Text → [PostLexer] → Lark Parse Tree → LarkElement Tree → Python Dict/JSON Reverse: Python Dict/JSON → LarkElement Tree → Lark Tree → HCL2 Text Direct: HCL2 Text → [PostLexer] → Lark Parse Tree → LarkElement Tree → Lark Tree → HCL2 Text ``` -------------------------------- ### Chain property access with builtins Source: https://github.com/amplify-education/python-hcl2/blob/main/docs/04_hq.md Combine accessing block labels with the 'length' builtin to count the number of labels. ```sh # Chain with builtins hq 'resource[*] | .labels | length' main.tf --value ``` -------------------------------- ### List SSM Parameter Paths Source: https://github.com/amplify-education/python-hcl2/blob/main/docs/05_hq_examples.md Extracts the names of AWS SSM parameters, useful for discovering parameter paths. ```sh hq 'resource.aws_ssm_parameter~[*] | .name | .value' infra/ --value ``` -------------------------------- ### Object Construction with hq Source: https://github.com/amplify-education/python-hcl2/blob/main/docs/04_hq.md Extract multiple fields into a JSON object per result, similar to jq syntax. Supports shorthand and renamed keys. ```sh # Shorthand — field name = key name hq 'module~[*] | {source, cpu, memory}' dir/ --json ``` ```sh # Renamed keys hq 'resource[*] | {type: .block_type, name: .name_labels}' main.tf --json ``` ```sh # Combine with select hq 'resource~[select(has("tags"))] | {name: .name_labels, tags}' main.tf --json ``` -------------------------------- ### Parse HCL2 to LarkElement Tree Source: https://context7.com/amplify-education/python-hcl2/llms.txt Utilize `hcl2.parses` or `hcl2.parse` to obtain a typed `StartRule` (LarkElement tree) for inspection or custom serialization. Pass `discard_comments=True` to remove comments. Requires the `hcl2` library. ```python import hcl2 from hcl2 import SerializationOptions text = 'x = 1\ny = "hello"\n' # Parse to typed IR tree tree = hcl2.parses(text) # Serialize with custom options from the tree data = hcl2.serialize(tree, serialization_options=SerializationOptions(with_meta=True)) # Parse discarding comments tree_no_comments = hcl2.parses(text, discard_comments=True) # From file with open("main.tf") as f: tree = hcl2.parse(f) ``` -------------------------------- ### Eval Mode Queries Source: https://github.com/amplify-education/python-hcl2/blob/main/docs/04_hq.md Treat the entire query as a Python expression using the -e flag. 'doc' is bound to the DocumentView. Note that some operations are blocked for safety. ```sh hq -e 'doc.blocks("variable")[0].attribute("default").value' variables.tf --json ``` ```sh hq -e 'sorted(doc.blocks("variable"), key=lambda b: b.name_labels[0])' variables.tf ``` ```sh hq -e 'list(filter(lambda b: b.attribute("default"), doc.blocks("variable")))' variables.tf ``` ```sh hq -e 'doc.find_by_predicate(lambda n: n.type == "attribute" and n.name == "ami")' main.tf ``` -------------------------------- ### Convert JSON to HCL2 using jsontohcl2 Source: https://github.com/amplify-education/python-hcl2/blob/main/docs/01_getting_started.md Use `jsontohcl2` to convert JSON files to HCL2 format. It supports various input sources like files, directories, glob patterns, and stdin. Output can be directed to stdout, a file, or a directory. Options include diffing, dry runs, and fragment conversion. ```sh jsontohcl2 output.json ``` ```sh jsontohcl2 output.json -o main.tf ``` ```sh jsontohcl2 output/ -o terraform/ ``` ```sh jsontohcl2 --diff original.tf modified.json ``` ```sh jsontohcl2 --semantic-diff original.tf modified.json ``` ```sh jsontohcl2 --semantic-diff original.tf --diff-json m.json ``` ```sh jsontohcl2 --dry-run file.json ``` ```sh jsontohcl2 --fragment - ``` ```sh echo '{"x": 1}' | jsontohcl2 ``` -------------------------------- ### List Provider Version Pins Source: https://github.com/amplify-education/python-hcl2/blob/main/docs/05_hq_examples.md Retrieves the required provider versions specified in the Terraform configuration, outputting in ndjson format. ```sh hq 'terraform.required_providers' infra/ --ndjson ``` -------------------------------- ### Traverse HCL2 IR Tree with hcl2.walk Module Source: https://context7.com/amplify-education/python-hcl2/llms.txt Utilize functions from the `hcl2.walk` module to traverse the Intermediate Representation (IR) tree of parsed HCL data. This includes semantic rule traversal, finding all nodes of a specific type, and navigating the ancestor chain. ```python import hcl2 from hcl2.walk import walk, walk_rules, walk_semantic, find_all, find_first, ancestors from hcl2.rules.base import AttributeRule, BlockRule text = ''' resource "aws_instance" "web" { ami = "ami-123" } resource "aws_s3_bucket" "logs" { bucket = "my-logs" } x = 1 ''' tree = hcl2.parses(text) # Find all AttributeRule nodes attrs = list(find_all(tree, AttributeRule)) print(len(attrs)) # 3 (ami, bucket, x) # Find first BlockRule first_block = find_first(tree, BlockRule) print(first_block) # BlockRule for aws_instance # Walk semantic rules only (skip NewLineOrCommentRule) for rule in walk_semantic(tree): print(type(rule).__name__) # Walk up the ancestor chain from a found node for parent in ancestors(attrs[0]): print(type(parent).__name__) ``` -------------------------------- ### Convert HCL2 to JSON using CLI Source: https://github.com/amplify-education/python-hcl2/blob/main/docs/01_getting_started.md Utilize the `hcl2tojson` CLI tool to convert HCL2 files or directories to JSON. Supports various input methods like files, directories, glob patterns, and stdin. ```sh hcl2tojson main.tf # single file to stdout ``` ```sh hcl2tojson main.tf -o output.json # single file to output file ``` ```sh hcl2tojson terraform/ -o output/ # directory to output dir ``` ```sh hcl2tojson --ndjson terraform/ # directory to stdout (NDJSON) ``` ```sh hcl2tojson --ndjson 'modules/**/*.tf' # glob + NDJSON streaming ``` ```sh hcl2tojson a.tf b.tf -o output/ # multiple files to output dir ``` ```sh hcl2tojson --only resource,module main.tf # block type filtering ``` ```sh hcl2tojson --fields cpu,memory main.tf # field projection ``` ```sh hcl2tojson --compact main.tf # single-line JSON ``` ```sh echo 'x = 1' | hcl2tojson # stdin (no args needed) ``` -------------------------------- ### Convert JSON directory to HCL2 directory Source: https://context7.com/amplify-education/python-hcl2/llms.txt Converts all JSON files in a directory to HCL2 files in a specified output directory. ```bash jsontohcl2 output/ -o terraform/ ``` -------------------------------- ### Piping hq to jq for Field Extraction into Arrays Source: https://github.com/amplify-education/python-hcl2/blob/main/docs/04_hq.md Extract specific fields like block type and name from resources using hq and reshape them into arrays with jq. ```sh # Reshaping — extract specific fields into arrays hq 'resource[*]' main.tf --json | jq '[.block_type, .name]' ``` -------------------------------- ### Builder.block() Method Signature Source: https://github.com/amplify-education/python-hcl2/blob/main/docs/03_advanced_api.md The `block` method on the `Builder` class creates a new block within the HCL document. It returns a child `Builder` instance, enabling chained calls for nested structures. ```python block( block_type: str, labels: Optional[List[str]] = None, __nested_builder__: Optional[Builder] = None, **attributes, ) -> Builder ```