PyLingual

https://github.com/syssec-utd/pylingual
Admin
A CPython bytecode decompiler supporting Python versions 3.6-3.13, optimized for readability and...
Tokens:87,215
Snippets:606
Trust Score:7.6
License:GPL-3.0
Update:1 week ago
Show doc for...
Tokens:
Raw
### Install PyLingual from source using uv

Source: https://github.com/syssec-utd/pylingual/blob/main/README.md

Clone the repository and install PyLingual using the uv package manager. This is the recommended setup method.

```sh
git clone https://github.com/syssec-utd/pylingual
uv tool install ./pylingual
```

--------------------------------

### Inst Usage Example

Source: https://github.com/syssec-utd/pylingual/blob/main/_autodocs/api-reference/editable-bytecode.md

Example demonstrating how to iterate over instructions in a PYCFile and use Inst properties and methods.

```APIDOC
## Inst Usage Example

### Description
Example usage of the Inst class.

### Code
```python
from pylingual.editable_bytecode import PYCFile

pyc = PYCFile("program.pyc")
for inst in pyc:
    if inst.is_jump:
        print(inst.opname, "->", inst.target.offset)
    if inst.starts_line:
        print(f"# line {inst.starts_line}")
    print(inst.get_dis_view())
```
```

--------------------------------

### Example Usage

Source: https://github.com/syssec-utd/pylingual/blob/main/_autodocs/api-reference/segmentation.md

Demonstrates how to use sliding_window and merge functions together.

```APIDOC
## Example

### Description
Example usage of `sliding_window` and `merge` from the `pylingual.segmentation.sliding_window` module.

### Code
```python
from pylingual.segmentation.sliding_window import sliding_window, merge

# instructions: list of (Inst, token_len)
windows = list(sliding_window(instructions, max_window_size=512, step_size=128))
# run each window through the segmentation model ...
results = merge(
    [((0, i),) for i in range(len(windows))],
    model_predictions,
    [idx for _, idx in windows],
    max_window_size=512,
    step=128,
)
```
```

--------------------------------

### Example — masking and restoring source

Source: https://github.com/syssec-utd/pylingual/blob/main/_autodocs/api-reference/masking.md

Example showing how to mask and restore source code using the pylingual masking utilities.

```APIDOC
## Example — masking and restoring source

### Description
Example demonstrating how to mask a source file and restore the masked source text after processing.

### Code Example
```python
from pathlib import Path
from pylingual.editable_bytecode import PYCFile
from pylingual.masking.model_disasm import create_global_masker, mask_source, restore_masked_source_text
from pylingual.utils.version import PythonVersion

pyc = PYCFile("program.pyc")
masker = create_global_masker(pyc)

masked = mask_source(Path("program.py"), masker, PythonVersion(3, 11))
# ... model runs on `masked` ...
restored_lines = restore_masked_source_text(masked.splitlines(), masker)
print("\n".join(restored_lines))
```
```

--------------------------------

### Example: Iterating over a PYCFile

Source: https://github.com/syssec-utd/pylingual/blob/main/_autodocs/api-reference/editable-bytecode.md

Shows how to load a .pyc file and iterate over its instructions, printing jump targets, line numbers, and disassembly views.

```python
from pylingual.editable_bytecode import PYCFile

pyc = PYCFile("program.pyc")
for inst in pyc:
    if inst.is_jump:
        print(inst.opname, "->", inst.target.offset)
    if inst.starts_line:
        print(f"# line {inst.starts_line}")
    print(inst.get_dis_view())
```

--------------------------------

### Example usage of Preprocessor

Source: https://github.com/syssec-utd/pylingual/blob/main/_autodocs/api-reference/preprocessor.md

Shows how to instantiate Preprocessor and preprocess a PYC file, converting container literals to single LOAD_CONST instructions.

```python
from pylingual.editable_bytecode import PYCFile
from pylingual.preprocessor import Preprocessor

pyc = PYCFile("program.pyc")
Preprocessor().preprocess(pyc)
# container literals are now single LOAD_CONST instructions
```

--------------------------------

### Example: load, patch, and save a .pyc file

Source: https://github.com/syssec-utd/pylingual/blob/main/_autodocs/api-reference/editable-bytecode.md

Shows how to load a .pyc file, print its version and name, apply a patch to remove docstrings, and save the patched copy.

```python
from pathlib import Path
from pylingual.editable_bytecode import PYCFile

pyc = PYCFile(Path("program.pyc"))
print(pyc.version, pyc.name)

# remove all docstring loads/stores, then save a patched copy
from pylingual.editable_bytecode.bytecode_patches import remove_docstrings
pyc.apply_patches([remove_docstrings])
pyc.save("patched.pyc")
```

--------------------------------

### Example usage of sliding_window and merge

Source: https://github.com/syssec-utd/pylingual/blob/main/_autodocs/api-reference/segmentation.md

Shows how to create windows with sliding_window and merge the results with merge, using max_window_size=512 and step_size=128.

```python
from pylingual.segmentation.sliding_window import sliding_window, merge

# instructions: list of (Inst, token_len)
windows = list(sliding_window(instructions, max_window_size=512, step_size=128))
# run each window through the segmentation model ...
results = merge(
    [((0, i),) for i in range(len(windows))],
    model_predictions,
    [idx for _, idx in windows],
    max_window_size=512,
    step=128,
)
```

--------------------------------

### Start Redis for translation cache with Docker Compose

Source: https://github.com/syssec-utd/pylingual/blob/main/README.md

Starts the bundled Redis service from the cloned repository using Docker Compose. The service listens on 127.0.0.1:6379. If Redis is already running there, reuse it instead.

```sh
cd pylingual
docker compose up -d redis
```

--------------------------------

### Example — alternate segmentations

Source: https://github.com/syssec-utd/pylingual/blob/main/_autodocs/api-reference/segmentation.md

Shows how to use get_top_k_predictions with a strategy built from m_deep_top_k and naive_confidence_priority to generate alternate segmentations, skipping the original (index 0) and printing those starting with 'B'.

```python
from functools import partial
from pylingual.segmentation.segmentation_search_strategies import (
    get_top_k_predictions,
    m_deep_top_k,
    naive_confidence_priority,
)

# predicted_boundary: [{entity, score}, ...] for one code object
strategy = partial(m_deep_top_k, priority_function=naive_confidence_priority, m=2, k=11)
alternates = get_top_k_predictions(strategy, predicted_boundary)
for alt in alternates[1:]:   # skip index 0 == original
    if alt[0] == "B":
        print(alt)
```

--------------------------------

### Example: Using Masker to print model views

Source: https://github.com/syssec-utd/pylingual/blob/main/_autodocs/api-reference/masking.md

Loads a PYC file, creates a global masker, and prints the model view for each instruction.

```python
from pylingual.editable_bytecode import PYCFile
from pylingual.masking.model_disasm import create_global_masker

pyc = PYCFile("program.pyc")
masker = create_global_masker(pyc)
for inst in pyc.instructions:
    print(masker.get_model_view(inst))

```

--------------------------------

### Decompiler usage example

Source: https://github.com/syssec-utd/pylingual/blob/main/_autodocs/api-reference/decompiler.md

Example showing how to decompile a .pyc file and print the decompiled source and success rate.

```python
from pylingual import decompile

result = decompile("example.pyc")
print(result.decompiled_source)
print(result.version, result.calculate_success_rate())
```

--------------------------------

### pylingual CLI usage examples

Source: https://github.com/syssec-utd/pylingual/blob/main/_autodocs/api-reference/cli.md

Shows three example invocations of the pylingual command: decompiling a single file, decompiling a directory tree with Redis caching, and pinning a version with force overwrite and quiet mode.

```sh
# decompile a single file
pylingual example.pyc

# decompile a whole directory tree, mirroring structure, with Redis caching
pylingual -o out --redis-host 127.0.0.1 --redis-port 6379 build/

# pin a version, overwrite outputs, and silence progress
pylingual -v 3.9 --force --quiet legacy.pyc
```

--------------------------------

### Example: loading models with custom config and Redis

Source: https://github.com/syssec-utd/pylingual/blob/main/_autodocs/api-reference/models.md

Example usage of load_models with a custom config file, Python version 3.11, and Redis caching enabled. Prints the types of the returned segmenter and translator.

```python
from pathlib import Path
from pylingual.models import load_models
from pylingual.utils.version import PythonVersion

segmenter, translator = load_models(
    config_file=Path("my_config.yaml"),
    version=PythonVersion(3.11),
    redis_cache_server_ip="127.0.0.1",
    redis_port=6379,
)
print(type(segmenter).__name__, type(translator).__name__)

```

--------------------------------

### Example - visualizing reconstruction

Source: https://github.com/syssec-utd/pylingual/blob/main/_autodocs/api-reference/control-flow-reconstruction.md

Enables CFG graphing to PNG files in the specified directory, then runs decompile() to write every CFG iteration to that directory.

```python
from pylingual.control_flow_reconstruction.cfg import CFG

CFG.enable_graphing("out/graphs", fmt="png")
# then run decompile() — every CFG iteration is written to out/graphs
```

--------------------------------

### Quick start: decompile using the command-line interface

Source: https://github.com/syssec-utd/pylingual/blob/main/_autodocs/api-reference/README.md

Run the `pylingual` console script with output directory `-o out`, Redis host `--redis-host 127.0.0.1`, and the input `.pyc` file.

```sh
pylingual -o out --redis-host 127.0.0.1 example.pyc
```

--------------------------------

### Example: reconstructing source directly

Source: https://github.com/syssec-utd/pylingual/blob/main/_autodocs/api-reference/control-flow-reconstruction.md

Shows how to create a SourceContext and print the final decompiled source. Requires a PYCFile, the unmasked translated lines indexed by starts_line - 1, and a mapping of each bytecode's code object to its control flow template.

```python
from pylingual.editable_bytecode import PYCFile
from pylingual.control_flow_reconstruction.structure import bc_to_cft
from pylingual.control_flow_reconstruction.source import SourceContext

pyc = PYCFile("program.pyc")
# `lines` are the unmasked translated statements, indexed by starts_line - 1
cfts = {bc.codeobj: bc_to_cft(bc, lines) for bc in pyc.iter_bytecodes()}
source_context = SourceContext(pyc, lines, cfts)
print(str(source_context))
```

--------------------------------

### CacheTranslator Example

Source: https://github.com/syssec-utd/pylingual/blob/main/_autodocs/api-reference/models.md

Example of direct translation with Redis cache using the CacheTranslator class.

```APIDOC
## CacheTranslator Example

### Description
Example of direct translation with Redis cache using the CacheTranslator class.

### Code Example
```python
from pylingual.models import CacheTranslator

# `model` and `tokenizer` come from load_models()
translator = CacheTranslator(
    model=model,
    tokenizer=tokenizer,
    python_version="3.11",
    redis_cache_server_ip="127.0.0.1",
    redis_port=6379,
    maxsize=10000,
)

statements = translator(["LOAD_CONST 0 , <mask_0> <SEP> STORE_FAST 1 , <mask_1>"])
print(statements[0])
```

```

--------------------------------

### PythonVersion usage example

Source: https://github.com/syssec-utd/pylingual/blob/main/_autodocs/api-reference/version.md

Example usage of PythonVersion: constructing from a string, checking tuple conversion, support status, and comparisons. Also demonstrates that unsupported versions normalize fine but is_supported() returns False, and that invalid strings raise ValueError with the message "version not-a-version is invalid".

```python
from pylingual.utils.version import PythonVersion

v = PythonVersion("3.10")
assert v.as_tuple() == (3, 10)
assert v.is_supported()
assert v >= "3.9"

try:
    PythonVersion((3, 99))  # normalizes fine to (3, 99), but 3.99 is not "supported"
    print("normalized; supported?", PythonVersion((3, 99)).is_supported())  # False
except ValueError:
    pass

try:
    PythonVersion("not-a-version")
except ValueError as e:
    print(e)  # "version not-a-version is invalid"
```

--------------------------------

### Example: iterating over equivalence results

Source: https://github.com/syssec-utd/pylingual/blob/main/_autodocs/api-reference/equivalence-check.md

Shows how to use decompile() and iterate over equivalence_results, printing OK for successful comparisons and FAIL with message and line number for failures.

```python
from pylingual import decompile

result = decompile("example.pyc")
for tr in result.equivalence_results:
    if tr.success:
        print(f"OK  {tr.names()}")
    else:
        print(f"FAIL {tr.names()}: {tr.message} at line {tr.failed_line_number}")
```

--------------------------------

### Example usage of compare_pyc

Source: https://github.com/syssec-utd/pylingual/blob/main/_autodocs/api-reference/equivalence-check.md

Shows how to call compare_pyc with two file paths and iterate over the results, printing the names, success status, and message for each TestResult.

```python
from pathlib import Path
from pylingual.equivalence_check import compare_pyc

results = compare_pyc(Path("original.pyc"), Path("recompiled.pyc"))
for r in results:
    print(r.names(), "Success" if r.success else "Failure", r.message)
```

--------------------------------

### Direct translation with Redis cache example

Source: https://github.com/syssec-utd/pylingual/blob/main/_autodocs/api-reference/models.md

Shows how to instantiate CacheTranslator with a Redis cache server and translate a statement. The model and tokenizer are assumed to come from load_models().

```python
from pylingual.models import CacheTranslator

# `model` and `tokenizer` come from load_models()
translator = CacheTranslator(
    model=model,
    tokenizer=tokenizer,
    python_version="3.11",
    redis_cache_server_ip="127.0.0.1",
    redis_port=6379,
    maxsize=10000,
)

statements = translator(["LOAD_CONST 0 , <mask_0> <SEP> STORE_FAST 1 , <mask_1>"])
print(statements[0])
```

--------------------------------

### CFG.ordered_iter

Source: https://github.com/syssec-utd/pylingual/blob/main/_autodocs/api-reference/control-flow-reconstruction.md

DFS post-order from start, sorting neighbors by offset descending, using a cached dominator tree.

```APIDOC
## CFG.ordered_iter

### Description
DFS post-order from `start`, sorting neighbors by offset descending, using a cached dominator tree.

### Method
Instance method

### Signature
```python
def ordered_iter(self):
```

### Parameters
None

### Returns
Iterator over nodes in DFS post-order.
```

--------------------------------

### Quick start: decompile a .pyc file using the library

Source: https://github.com/syssec-utd/pylingual/blob/main/_autodocs/api-reference/README.md

Import the `decompile` function from `pylingual`, decompile a `.pyc` file, and print the decompiled source and success rate. The `save_to` parameter writes the output to a file.

```python
from pylingual import decompile

result = decompile("example.pyc", save_to="example.py")
print(result.decompiled_source)
print(f"success rate: {result.calculate_success_rate():.2%}")
```

--------------------------------

### lazy_import usage example from models.py

Source: https://github.com/syssec-utd/pylingual/blob/main/_autodocs/api-reference/bytecode-utils.md

Shows how to use lazy_import to delay importing torch, transformers, and huggingface_hub in an else branch.

```python
else:
    lazy_import("torch")
    lazy_import("transformers")
    lazy_import("huggingface_hub")
```

--------------------------------

### Masking and restoring source code with create_global_masker

Source: https://github.com/syssec-utd/pylingual/blob/main/_autodocs/api-reference/masking.md

Shows how to mask a Python source file using a global masker created from a PYC file, then restore the masked lines. Requires a PYC file and a Python version (3.11 in the example).

```python
from pathlib import Path
from pylingual.editable_bytecode import PYCFile
from pylingual.masking.model_disasm import create_global_masker, mask_source, restore_masked_source_text
from pylingual.utils.version import PythonVersion

pyc = PYCFile("program.pyc")
masker = create_global_masker(pyc)

masked = mask_source(Path("program.py"), masker, PythonVersion(3, 11))
# ... model runs on `masked` ...
restored_lines = restore_masked_source_text(masked.splitlines(), masker)
print("\n".join(restored_lines))
```

--------------------------------

### Inst Methods

Source: https://github.com/syssec-utd/pylingual/blob/main/_autodocs/api-reference/editable-bytecode.md

Methods of the Inst class for refreshing argument representations, managing dependencies, getting next instructions, computing stack effects, and creating Inst objects from xdis instructions.

```APIDOC
## Inst Methods

### Description
Methods available on Inst instances.

### Methods

#### refresh_argrepr
- **Signature**: `def refresh_argrepr(self):`
- **Description**: Recomputes argval/argrepr from the current target offset.

#### add_reqs
- **Signature**: `def add_reqs(self, *reqs):`
- **Description**: Records dependency requirements (populates deps on the targets).

#### get_next_instructions
- **Signature**: `def get_next_instructions(self, follow_uncond_jumps: bool = False):`
- **Description**: Robust next-instruction set; optionally follows unconditional jumps.

#### get_stack_effect
- **Signature**: `def get_stack_effect(self, jump: bool = True):`
- **Description**: Stack effect via xdis.cross_dis.xstack_effect (may be None).

#### get_dis_view
- **Signature**: `def get_dis_view(self) -> str:`
- **Description**: Dis-like string of the instruction.

#### from_instruction
- **Signature**: `@classmethod def from_instruction(self, bytecode, inst):`
- **Description**: Creates an Inst from an xdis Instruction.

### Response
#### Success Response
Returns the result of the method as described.

#### Response Example
```python
inst.refresh_argrepr()
inst.add_reqs(req1, req2)
next_insts = inst.get_next_instructions()
stack_effect = inst.get_stack_effect()
dis_view = inst.get_dis_view()
new_inst = Inst.from_instruction(bytecode, xdis_inst)
```
```

--------------------------------

### Generate dataset with prepare_dataset.py

Source: https://github.com/syssec-utd/pylingual/blob/main/dev_scripts/README.md

Run this command to create the dataset from a JSON file. The JSON file should be based on the sample `sample_jsons/py36-sample-data.json`.

```bash
uv run prepare_dataset.py <path to JSON>
```

--------------------------------

### fix_format_string_lno

Source: https://github.com/syssec-utd/pylingual/blob/main/_autodocs/api-reference/editable-bytecode.md

Aligns format-string LOAD_CONST line starts.

```APIDOC
## fix_format_string_lno

### Description
Aligns format-string `LOAD_CONST` line starts.

### Method
`def fix_format_string_lno(self):`

### Parameters
None

### Returns
None
```

--------------------------------

### filter_subwords

Source: https://github.com/syssec-utd/pylingual/blob/main/_autodocs/api-reference/segmentation.md

Filters token-classification results to those whose word does not start with '##' (i.e., drops subword pieces).

```APIDOC
## filter_subwords

### Description
Filters token-classification results to those whose `word` does not start with `"##"` (i.e., drops subword pieces).

### Method
Function

### Signature
```python
def filter_subwords(results):
```

### Parameters
- **results** - The token-classification results to filter.

### Returns
- Filtered results containing only words that do not start with `"##"`.
```

--------------------------------

### make_relative

Source: https://github.com/syssec-utd/pylingual/blob/main/_autodocs/api-reference/editable-bytecode.md

Returns a [JUMP_FORWARD] instruction list.

```APIDOC
## make_relative

### Description
Returns a `[JUMP_FORWARD]` instruction list.

### Method
`def make_relative(self, offset, target):`

### Parameters
- `offset` - The current offset.
- `target` - The jump target.

### Returns
A list containing a `JUMP_FORWARD` instruction.
```

--------------------------------

### PyenvError class definition

Source: https://github.com/syssec-utd/pylingual/blob/main/_autodocs/errors.md

Defines the PyenvError exception. Raised when pyenv is not on PATH or has no installed version for the target.

```python
class PyenvError(Exception):
    pass
```

--------------------------------

### SourceContext.__init__

Source: https://github.com/syssec-utd/pylingual/blob/main/_autodocs/api-reference/control-flow-reconstruction.md

Initializes a SourceContext with a PYCFile, translated lines, and a mapping of code objects to control flow templates.

```APIDOC
## SourceContext.__init__

### Description
Initializes a SourceContext with a PYCFile, translated lines, and a mapping of code objects to control flow templates.

### Method
__init__

### Signature
`def __init__(self, pyc: PYCFile, lines: list[str], cfts: dict[Code3, ControlFlowTemplate])`

### Parameters
- **pyc** (PYCFile) - Required - The PYCFile object.
- **lines** (list[str]) - Required - The sanitized translated lines.
- **cfts** (dict[Code3, ControlFlowTemplate]) - Required - Mapping of code objects to their root control flow templates.

```

--------------------------------

### Basic usage of decompile()

Source: https://github.com/syssec-utd/pylingual/blob/main/_autodocs/api-reference/decompiler.md

Shows how to decompile a .pyc file using the default settings and print the decompiled source and success rate.

```python
from pathlib import Path
from pylingual import decompile

result = decompile(pyc=Path("program.pyc"))

print(result.decompiled_source)
print(f"Code object success rate: {result.calculate_success_rate():.2%}")
```

--------------------------------

### Decompiler.run_cflow_reconstruction

Source: https://github.com/syssec-utd/pylingual/blob/main/_autodocs/api-reference/decompiler.md

Builds {codeobj: bc_to_cft(...)} over a tracked list, constructs a SourceContext, and sets the four header SourceLines.

```APIDOC
## Decompiler.run_cflow_reconstruction

### Description
Builds `{codeobj: bc_to_cft(...)}` over a tracked list, constructs a `SourceContext`, and sets the four header `SourceLine`s (PyLingual banner, internal filename, bytecode version + magic, source timestamp).
```

--------------------------------

### PyLingual CLI usage and options

Source: https://github.com/syssec-utd/pylingual/blob/main/README.md

Displays the full command-line help for PyLingual, including options for output directory, config file, Python version, top-k, Redis host/port, quiet mode, trust-lnotab, and init-pyenv.

```text
Usage: pylingual [OPTIONS] [FILES]...

  End to end pipeline to decompile Python bytecode into source code.

Options:
  -o, --out-dir PATH      The directory to export results to.
  -c, --config-file PATH  Config file for model information.
  -v, --version VERSION   Python version of the .pyc, default is auto
                          detection.
  -k, --top-k INT         Maximum number of additional segmentations to
                          consider.
  -r, --redis-host HOST  Redis translation cache host; defaults to
                         $PYLINGUAL_REDIS_HOST. Unset disables caching.
  --redis-port PORT      Redis port; defaults to $PYLINGUAL_REDIS_PORT
                         or 6379.
  -q, --quiet             Suppress console output.
  --trust-lnotab          Use the lnotab for segmentation instead of the
                          segmentation model.
  --init-pyenv            Install pyenv before decompiling.
  -h, --help              Show this message and exit.
```

--------------------------------

### filter_subwords function definition

Source: https://github.com/syssec-utd/pylingual/blob/main/_autodocs/api-reference/segmentation.md

Defines filter_subwords, which filters token-classification results to those whose 'word' does not start with '##' (drops subword pieces).

```python
def filter_subwords(results):
```

--------------------------------

### CFG.from_graph static method

Source: https://github.com/syssec-utd/pylingual/blob/main/_autodocs/api-reference/control-flow-reconstruction.md

Wraps a raw instruction CFG, runs template matching, annotates source lines, converts edge types, adds meta nodes, and optionally iterates.

```python
@staticmethod
def from_graph(cfg: nx.DiGraph, bytecode: EditableBytecode, iterate=True, source=None) -> CFG:
```

--------------------------------

### normalize_masks

Source: https://github.com/syssec-utd/pylingual/blob/main/_autodocs/api-reference/masking.md

Renumbers <mask_N> tokens in a statement to start at 0, compacting gaps, and returns the renumbered statement along with the original mask order.

```APIDOC
## normalize_masks

### Description
Renumbers `<mask_N>` tokens in a statement to start at 0 (compacting gaps) and returns `(renumbered_statement, mask_order)` where `mask_order[i]` is the original mask id for renumber `i`.

### Signature
```python
def normalize_masks(statement: str) -> tuple[str, list[str]]:
```

### Parameters
- **statement** (str) - Required - The statement containing mask tokens.

### Returns
- **tuple[str, list[str]]** - A tuple of the renumbered statement and the list of original mask ids in order.
```

--------------------------------

### Inst.__init__ signature

Source: https://github.com/syssec-utd/pylingual/blob/main/_autodocs/api-reference/editable-bytecode.md

Constructor for Inst. Takes the containing EditableBytecode, opcode metadata, and optional jump target. Fields include dependency-graph attributes (deps, reqs, stack, pop) and exception-table markers for 3.11+.

```python
def __init__(
    self,
    bytecode: "EditableBytecode",
    opname: str,
    opcode,
    optype: str,
    inst_size: int,
    arg: int,
    argval: "Any",
    argrepr: str,
    has_arg: bool,
    offset: int,
    starts_line: int,
    is_jump_target: bool,
    has_extended_arg: bool,
    target: Optional["Inst"] = None,
):
```

--------------------------------

### CFG.enable_graphing static method

Source: https://github.com/syssec-utd/pylingual/blob/main/_autodocs/api-reference/control-flow-reconstruction.md

Enables CFG visualization by writing DOT/graphviz images per iteration into the given path; requires pydot.

```python
@staticmethod
def enable_graphing(graph_path: Path | str, fmt: str = "jpg"):
```

--------------------------------

### SourceContext.init_header

Source: https://github.com/syssec-utd/pylingual/blob/main/_autodocs/api-reference/control-flow-reconstruction.md

Adds docstring, generator yield placeholder, and global/nonlocal header lines to each template.

```APIDOC
## SourceContext.init_header

### Description
Adds docstring, `if False: yield` (for generators without yield), and `global`/`nonlocal` header lines to each template.

### Method
init_header

### Signature
`def init_header(self):`

### Parameters
None

### Returns
None

```

--------------------------------

### first_instruction

Source: https://github.com/syssec-utd/pylingual/blob/main/_autodocs/api-reference/editable-bytecode.md

The first instruction in the bytecode.

```APIDOC
## first_instruction

### Description
`self.instructions[0]`.

### Property
`Inst`
```

--------------------------------

### get_top_k_predictions function definition

Source: https://github.com/syssec-utd/pylingual/blob/main/_autodocs/api-reference/segmentation.md

Defines get_top_k_predictions, which converts a predicted_boundary list of {entity, score} dicts to a bitmap, applies the strategy to get transformations, XORs them with the initial segmentation, and converts back to entity strings.

```python
def get_top_k_predictions(strategy: SearchStrategy, predicted_boundary: List[Dict]) -> List[List[str]]:
```

--------------------------------

### Version pin, timeout, and error handling with decompile()

Source: https://github.com/syssec-utd/pylingual/blob/main/_autodocs/api-reference/decompiler.md

Demonstrates decompiling with explicit version, timeout, and error handling for TimeoutError, FileNotFoundError, and ValueError. Also shows iterating over equivalence_results.

```python
from pathlib import Path
from pylingual import decompile

try:
    result = decompile(
        pyc=Path("legacy.pyc"),
        save_to=Path("legacy.py"),
        version="3.7",          # skip auto-detection
        top_k=5,
        timeout=120,            # give up after 120 seconds
        trust_lnotab=False,
    )
except TimeoutError as e:
    print("Decompilation timed out:", e)
except FileNotFoundError as e:
    print("Config missing:", e)
except ValueError as e:
    print("Unsupported or malformed version:", e)

# equivalence_results holds one TestResult per code object
for tr in result.equivalence_results:
    print(tr)
```

--------------------------------

### get_top_k_predictions

Source: https://github.com/syssec-utd/pylingual/blob/main/_autodocs/api-reference/segmentation.md

Given a predicted_boundary list of {entity, score} dicts, converts entities to a bitmap, applies strategy(scores) to get transformations, XORs them with the initial segmentation, and converts back to entity strings. Returns list[list[str]] of alternate boundary entity sequences.

```APIDOC
## get_top_k_predictions

### Description
Given a `predicted_boundary` list of `{entity, score}` dicts, converts entities to a bitmap, applies `strategy(scores)` to get transformations, XORs them with the initial segmentation, and converts back to entity strings. Returns `list[list[str]]` of alternate boundary entity sequences.

### Method
Function

### Signature
```python
def get_top_k_predictions(strategy: SearchStrategy, predicted_boundary: List[Dict]) -> List[List[str]]:
```

### Parameters
- **strategy** (SearchStrategy) - Required - A callable that takes scores and returns a list of bitmaps (transformations).
- **predicted_boundary** (List[Dict]) - Required - A list of dictionaries with `entity` and `score` keys.

### Returns
- List[List[str]] - Alternate boundary entity sequences.
```

--------------------------------

### bc_to_cft

Source: https://github.com/syssec-utd/pylingual/blob/main/_autodocs/api-reference/control-flow-reconstruction.md

Builds the control-flow graph from an EditableBytecode and returns the root ControlFlowTemplate after structuring.

```APIDOC
## bc_to_cft

### Description
Builds the CFG from an `EditableBytecode` and returns the root `ControlFlowTemplate` after structuring.

### Function Signature
```python
def bc_to_cft(bc: EditableBytecode, source: list[str]):
```

### Parameters
- **bc** (EditableBytecode) - Required - The bytecode object to analyze.
- **source** (list[str]) - Required - The source code lines.

### Returns
- **ControlFlowTemplate** - The root template representing the structured control flow.
```

--------------------------------

### Segmentation boundary dict shape

Source: https://github.com/syssec-utd/pylingual/blob/main/_autodocs/types.md

Defines the exact shape of segmentation boundary dicts: {"entity": str, "score": float}. entity is one of "B" (boundary/statement start), "I" (inside), "E" (end). Produced by the segmentation pipeline and update_segmentation_from_lnotab; consumed by Decompiler.make_translation_request and get_top_k_predictions.

```python
{"entity": str, "score": float}
```

--------------------------------

### make_absolute

Source: https://github.com/syssec-utd/pylingual/blob/main/_autodocs/api-reference/editable-bytecode.md

Returns a [JUMP_ABSOLUTE] instruction list.

```APIDOC
## make_absolute

### Description
Returns a `[JUMP_ABSOLUTE]` instruction list.

### Method
`def make_absolute(self, offset, target):`

### Parameters
- `offset` - The current offset.
- `target` - The jump target.

### Returns
A list containing a `JUMP_ABSOLUTE` instruction.
```

--------------------------------

### create_arg_inst

Source: https://github.com/syssec-utd/pylingual/blob/main/_autodocs/api-reference/editable-bytecode.md

Builds a bare Inst with an argument.

```APIDOC
## create_arg_inst

### Description
Builds a bare `Inst` with an argument.

### Method
`def create_arg_inst(self, opname: str, arg, offset: int = -1):`

### Parameters
- `opname` (str) - The opcode name.
- `arg` - The argument value.
- `offset` (int, optional) - The offset. Defaults to -1.

### Returns
A new `Inst` instance.
```

--------------------------------

### Supporting utilities

Source: https://github.com/syssec-utd/pylingual/blob/main/_autodocs/api-reference/segmentation.md

Utility functions for bitstring generation, index conversion, windowing, and bitmap/entity conversions.

```APIDOC
## Supporting utilities

### Description
Utility functions for bitstring generation, index conversion, windowing, and bitmap/entity conversions.

### Functions

#### bitstring_index_generator

```python
def bitstring_index_generator(length: int, num_ones: int) -> Generator[Tuple[int], None, None]:
```

Incrementally yields index-tuples of bitstrings of length `length` with `num_ones` ones.

#### indices_to_num

```python
def indices_to_num(indices: List[int]) -> int:
```

Converts bit indices to a numeric key (`sum(2**i)`) for ordering.

#### sliding_window

```python
def sliding_window(iterable: Iterable, window=2) -> Iterable:
```

`itertools.pairwise`-style windowing over an iterable. Raises `ValueError("Window size must be at least 1")` for `window < 1`.

#### bitmap_to_entities

```python
def bitmap_to_entities(bitmap: BitMap) -> List[str]:
```

Maps `(current, next)` boolean pairs to `B`/`I`/`E` entity labels.

#### entities_to_bitmap

```python
def entities_to_bitmap(entities) -> BitMap:
```

Converts entity labels back to a numpy boolean bitmap (`entity == "B"`).
```

--------------------------------

### Decompiler.preprocess_bytecode

Source: https://github.com/syssec-utd/pylingual/blob/main/_autodocs/api-reference/decompiler.md

Runs Preprocessor().preprocess(self.pyc). Failures are logged and swallowed.

```APIDOC
## Decompiler.preprocess_bytecode

### Description
Runs `Preprocessor().preprocess(self.pyc)`. Failures are logged (`"Preprocessing failed for ... continuing without container recovery"`) and swallowed.
```

--------------------------------

### CacheTranslator.__init__

Source: https://github.com/syssec-utd/pylingual/blob/main/_autodocs/api-reference/models.md

Initializes a CacheTranslator instance with the given model, tokenizer, Python version, and optional Redis configuration. Moves the model to the specified device and sets up the LRU cache.

```APIDOC
## CacheTranslator.__init__

### Description
Initializes a CacheTranslator instance with the given model, tokenizer, Python version, and optional Redis configuration. Moves the model to the specified device and sets up the LRU cache.

### Method
Constructor

### Signature
```python
def __init__(self, model: transformers.T5ForConditionalGeneration, tokenizer: transformers.RobertaTokenizer, python_version: str, redis_cache_server_ip: str = None, redis_port: int = 1679, device="cpu", maxsize=50000):
```

### Parameters
- **model** (transformers.T5ForConditionalGeneration) - Required - The loaded statement translation model; moved to `device`.
- **tokenizer** (transformers.RobertaTokenizer) - Required - The tokenizer for the statement model.
- **python_version** (str) - Required - Version string used as the Redis namespace prefix (`f"{python_version}:"`).
- **redis_cache_server_ip** (str) - Optional - Redis host; enables the Redis cache when not `None`. Default: `None`.
- **redis_port** (int) - Optional - Redis port. Default: `1679`.
- **device** (str) - Optional - Torch device string; `model.to(self.device)` is called. Default: `"cpu"`.
- **maxsize** (int) - Optional - Maximum number of entries kept in the in-memory LRU cache. Default: `50000`.

### Public Attributes
- `model` - The loaded T5 model.
- `tokenizer` - The tokenizer.
- `device` - The device string.
- `cache` - An `OrderedDict` LRU cache.
- `maxsize` - Maximum cache size.
- `redis_enabled` (bool) - Whether Redis is enabled.
- `redis_db` (redis.StrictRedis) - Redis database instance (when enabled).
- `redis_namespace_prefix` - Redis namespace prefix (when enabled).

```

--------------------------------

### CFG.from_graph

Source: https://github.com/syssec-utd/pylingual/blob/main/_autodocs/api-reference/control-flow-reconstruction.md

Static factory method that wraps a raw instruction CFG, runs template matching, annotates source lines, converts edge types, adds meta nodes, and optionally iterates.

```APIDOC
## CFG.from_graph

### Description
Wraps a raw instruction CFG, runs `InstTemplate.match_all`, annotates `source_line` on instructions (when `source` is given), converts edge `type` strings to `EdgeKind`, adds the `start`/`end` meta nodes and meta edges, runs `BlockTemplate.match_all`, and (when `iterate=True`) calls `self.iterate()`.

### Method
Static method

### Signature
```python
@staticmethod
def from_graph(cfg: nx.DiGraph, bytecode: EditableBytecode, iterate=True, source=None) -> CFG:
```

### Parameters
- **cfg** (nx.DiGraph) - Required - The raw instruction CFG.
- **bytecode** (EditableBytecode) - Required - The bytecode object.
- **iterate** (bool) - Optional - Whether to run the iteration process. Defaults to True.
- **source** (optional) - Optional - Source code for line annotation.

### Returns
- **CFG** - The constructed CFG object.
```

--------------------------------

### sliding_window

Source: https://github.com/syssec-utd/pylingual/blob/main/_autodocs/api-reference/segmentation.md

Yields windows of instructions that fit within a maximum token size, stepping forward by a configurable step size.

```APIDOC
## `sliding_window`

### Description
Yields `(window, index_window)` tuples. Each window accumulates instructions until adding the next instruction would exceed `max_window_size` (token count), then the iterator steps forward by `step_size` tokens and starts a new window. The final (partially full) window is yielded at exhaustion.

### Signature
```python
def sliding_window(instructions: list, max_window_size: int, step_size: int = 1) -> tuple[list[str], list[int]]:
```

### Parameters
- **instructions** (list) - Required - List of `(inst, token_len)` pairs.
- **max_window_size** (int) - Required - Max tokens per window.
- **step_size** (int) - Optional - Tokens to step between windows (default `1`).

### Returns
Generator of `(window_instructions, window_indices)` tuples.
```

--------------------------------

### Inst.__init__

Source: https://github.com/syssec-utd/pylingual/blob/main/_autodocs/api-reference/editable-bytecode.md

Constructor for the Inst class. Initializes an instruction instance with all its attributes, including bytecode reference, opcode details, arguments, offset, and optional jump target.

```APIDOC
## Inst.__init__

### Description
Initializes an Inst object with the given parameters.

### Method
Constructor

### Parameters
- **bytecode** (EditableBytecode) - Required - The containing bytecode object.
- **opname** (str) - Required - The operation name.
- **opcode** - Required - The opcode value.
- **optype** (str) - Required - The operation type (e.g., 'jabs', 'jrel').
- **inst_size** (int) - Required - The instruction size in bytes.
- **arg** (int) - Required - The argument value.
- **argval** (Any) - Required - The argument value object.
- **argrepr** (str) - Required - The argument representation string.
- **has_arg** (bool) - Required - Whether the instruction has an argument.
- **offset** (int) - Required - The original offset.
- **starts_line** (int) - Required - The line number where the instruction starts.
- **is_jump_target** (bool) - Required - Whether this instruction is a jump target.
- **has_extended_arg** (bool) - Required - Whether the instruction has an extended argument.
- **target** (Optional[Inst]) - Optional - The jump target instruction.

### Request Example
```python
Inst(bytecode, opname, opcode, optype, inst_size, arg, argval, argrepr, has_arg, offset, starts_line, is_jump_target, has_extended_arg, target)
```

### Response
#### Success Response
Returns an Inst instance.

#### Response Example
```python
<Inst object>
```
```

--------------------------------

### Train statement model with train_models.py --statement

Source: https://github.com/syssec-utd/pylingual/blob/main/dev_scripts/README.md

Trains the statement model using a JSON configuration file based on `sample_jsons/py36-sample-statement.json`.

```bash
uv run train_models.py --statement <path to JSON>
```

--------------------------------

### Public exports in __init__.py

Source: https://github.com/syssec-utd/pylingual/blob/main/_autodocs/api-reference/editable-bytecode.md

The __all__ list defines the public API of the pylingual.editable_bytecode package.

```python
__all__ = ["EditableBytecode", "Inst", "PYCFile"]
```

--------------------------------

### CFG.enable_graphing

Source: https://github.com/syssec-utd/pylingual/blob/main/_autodocs/api-reference/control-flow-reconstruction.md

Static method that enables CFG visualization, writing DOT/graphviz images per iteration to the specified path. Requires pydot.

```APIDOC
## CFG.enable_graphing

### Description
Turns on CFG visualization (writes DOT/graphviz images per iteration into `graph_path`); requires `pydot`. After this, `CFG.visualize`/`CFG.layout_nodes` are active.

### Method
Static method

### Signature
```python
@staticmethod
def enable_graphing(graph_path: Path | str, fmt: str = "jpg"):
```

### Parameters
- **graph_path** (Path | str) - Required - Directory path where graph images will be written.
- **fmt** (str) - Optional - Image format (e.g., "jpg", "png"). Defaults to "jpg".

### Returns
None
```

--------------------------------

### new_instruction

Source: https://github.com/syssec-utd/pylingual/blob/main/_autodocs/api-reference/editable-bytecode.md

Builds an Inst bound to this bytecode without inserting it.

```APIDOC
## new_instruction

### Description
Builds an `Inst` bound to this bytecode without inserting it.

### Method
`def new_instruction(self, *args, **kwargs):`

### Parameters
- `*args` - Positional arguments for the instruction.
- `**kwargs` - Keyword arguments for the instruction.

### Returns
A new `Inst` instance bound to this bytecode.
```

--------------------------------

### PythonVersion Constructor

Source: https://github.com/syssec-utd/pylingual/blob/main/_autodocs/api-reference/version.md

Creates a PythonVersion instance from various input types (PythonVersion, int, float, str, tuple). Raises ValueError for invalid inputs.

```APIDOC
## PythonVersion Constructor

### Description
Creates a PythonVersion instance from various input types (PythonVersion, int, float, str, tuple). Raises ValueError for invalid inputs.

### Method
Constructor

### Signature
```python
def __init__(self, x):
```

### Parameters
- **x** (PythonVersion | int | float | str | tuple) - Required - The version to represent. Accepted forms: PythonVersion, int (interpreted as (3, x)), float (with 3.1 → "3.10" special case), str (e.g. "3.9" or "39"), or (major, minor) tuple.

### Errors
- **ValueError** - Raised when the value cannot be normalized to a (int, int) pair. Message: "version {x} is invalid"

### Notes
Constructing a version does not require it to be supported; is_supported() is a separate check.
```

--------------------------------

### PYCFile.__init__

Source: https://github.com/syssec-utd/pylingual/blob/main/_autodocs/api-reference/editable-bytecode.md

Constructs a PYCFile object from raw pyc bytes, a filesystem path, or an open file object. It extracts all bytecode recursively into an EditableBytecode tree.

```APIDOC
## PYCFile.__init__

### Description
Constructs a PYCFile object from raw pyc bytes, a filesystem path, or an open file object. It extracts all bytecode recursively into an EditableBytecode tree.

### Method
Constructor

### Endpoint
PYCFile.__init__(source, name_prefix=None)

### Parameters
#### Path Parameters
- **source** (bytes | pathlib.Path | file-like) - Required - Raw pyc bytes, a filesystem path, or an open file object. A Path also sets self.pyc_path.
- **name_prefix** (str | None) - Optional - Prefix applied to each code object's name (used for namespacing).

### Request Example
```python
from pylingual.editable_bytecode import PYCFile
pyc = PYCFile(Path("program.pyc"))
```

### Response
#### Success Response
- **pyc_path** (Path | None) - The path if source was a Path.
- **source** - The original source.
- **timestamp** - The pyc timestamp.
- **magic** - The pyc magic number.
- **code** - The root EditableBytecode object.
- **ispypy** (bool) - Whether the pyc is from PyPy.
- **source_size** - The source size.
- **sip_hash** - The SIP hash.
- **file_offsets** - File offsets.
- **version** (PythonVersion) - The Python version.

#### Error Handling
- **ValueError** - If the pyc's embedded version is not a supported 3.x version.
```

--------------------------------

### CFG.iterate

Source: https://github.com/syssec-utd/pylingual/blob/main/_autodocs/api-reference/control-flow-reconstruction.md

Bumps the iteration counter and calls visualize().

```APIDOC
## CFG.iterate

### Description
Bumps the iteration counter and calls `visualize()`.

### Method
Instance method

### Signature
```python
def iterate(self):
```

### Parameters
None

### Returns
None
```

--------------------------------

### recover

Source: https://github.com/syssec-utd/pylingual/blob/main/_autodocs/api-reference/preprocessor.md

Runs each strategy in order and returns the first non-None Recovery. If none applies, returns Recovery(None, False) (incomplete). Importing this module triggers strategy registration.

```APIDOC
## recover

### Description
Runs each strategy in order and returns the first non-`None` `Recovery`; if none applies, returns `Recovery(None, False)` (incomplete). Importing this module triggers strategy registration via `from . import strategies`.

### Signature
```python
def recover(seg: Segment, indent: int = 0) -> Recovery
```
```

--------------------------------

### iteration

Source: https://github.com/syssec-utd/pylingual/blob/main/_autodocs/api-reference/control-flow-reconstruction.md

Runs a single iteration of template matching over the CFG, trying templates in priority order per run.

```APIDOC
## iteration

### Description
For each run index, iterates over `cfg.ordered_iter()` and tries each template in the run's priority order; returns `True` as soon as any template matches.

### Function Signature
```python
def iteration(cfg: CFG, runs: list[list[type[ControlFlowTemplate]]]):
```

### Parameters
- **cfg** (CFG) - Required - The control-flow graph to iterate over.
- **runs** (list[list[type[ControlFlowTemplate]]]) - Required - A list of template runs, each a list of template classes in priority order.

### Returns
- **bool** - `True` if any template matched during this iteration, otherwise `False`.
```

--------------------------------

### SourceContext __init__ signature

Source: https://github.com/syssec-utd/pylingual/blob/main/_autodocs/api-reference/control-flow-reconstruction.md

Constructor signature for the SourceContext class. It takes a PYCFile object, a list of sanitized translated lines, and a dictionary mapping code objects to their control flow templates.

```python
def __init__(self, pyc: PYCFile, lines: list[str], cfts: dict[Code3, ControlFlowTemplate]):
```

--------------------------------

### PythonVersion Methods

Source: https://github.com/syssec-utd/pylingual/blob/main/_autodocs/api-reference/version.md

Methods for checking support, converting to string/float/tuple, and normalizing input values.

```APIDOC
## PythonVersion Methods

### Description
Methods for checking support, converting to string/float/tuple, and normalizing input values.

### Methods

#### is_supported
```python
def is_supported(self) -> bool:
```
Returns True if the version is in supported_tuples (3.6–3.15).

#### as_str
```python
def as_str(self):
```
Returns "major.minor" string.

#### as_float
```python
def as_float(self):
```
Returns float(str(self)), e.g. 3.9.

#### as_tuple
```python
def as_tuple(self):
```
Returns (major, minor) tuple.

#### normalize (static)
```python
@staticmethod
def normalize(x) -> tuple[int, int] | None:
```
Normalizes any accepted input to a (major, minor) tuple, or None if not possible.
```

--------------------------------

### PYCFile.__init__ signature

Source: https://github.com/syssec-utd/pylingual/blob/main/_autodocs/api-reference/editable-bytecode.md

Constructor signature for the PYCFile class. Accepts raw pyc bytes, a path, or a file-like object, and an optional name prefix.

```python
def __init__(self, source, name_prefix=None):
```
Tokens:
Raw