### Install CredSweeper from source

Source: https://credsweeper.readthedocs.io/en/latest/_sources/install.rst.txt

Development installation steps including cloning the repository and installing requirements.

```bash
git clone https://github.com/Samsung/CredSweeper.git
cd CredSweeper
# Annotate "onnxruntime" if you don't want to use the ML validation feature.
pip install -qr requirements.txt
```

--------------------------------

### Install pre-commit hooks

Source: https://credsweeper.readthedocs.io/en/latest/how_to_contribute.html

Install the pre-commit framework to ensure code quality before committing changes.

```bash
pip install pre-commit
pre-commit install

```

--------------------------------

### Install Git on Debian-based systems

Source: https://credsweeper.readthedocs.io/en/latest/_sources/install.rst.txt

Command to install the git dependency if missing.

```bash
sudo apt install git
```

--------------------------------

### CredSweeper CLI Text Output Example

Source: https://credsweeper.readthedocs.io/en/latest/_sources/guide.rst.txt

An example of the text-based output format from CredSweeper when run via the command line.

```text
rule: Password | severity: medium | confidence: moderate | ml_probability: 0.9149653911590576 | line_data_list: [path: tests/samples/password.gradle | line_num: 1 | value: 'cackle!' | line: 'password = "cackle!"']

```

--------------------------------

### Install CredSweeper via pip

Source: https://credsweeper.readthedocs.io/en/latest/_sources/install.rst.txt

Standard installation command for the CredSweeper package.

```bash
pip install credsweeper
```

--------------------------------

### Example Credential Line

Source: https://credsweeper.readthedocs.io/en/latest/_sources/overall_architecture.rst.txt

A sample line of code used to demonstrate the ML validation extraction process.

```python
my_db_password = "NUU423cds"
```

--------------------------------

### Build and Scan Docker Image with CredSweeper

Source: https://credsweeper.readthedocs.io/en/latest/_sources/guide.rst.txt

This example shows how to build a Docker image, save it, and then scan it for credentials using CredSweeper with deep scanning enabled.

```docker
FROM scratch
ADD tests/samples /

```

```bash
docker build . --tag test_samples
docker save test_samples --output test_samples.docker
python -m credsweeper --path test_samples.docker --save-json output.json --depth 3

```

--------------------------------

### PEM Key Ignore Starts

Source: https://credsweeper.readthedocs.io/en/latest/credsweeper.utils.html

List of strings that indicate lines to be ignored at the beginning of a potential PEM key block.

```python
ignore_starts _ = ['-----BEGIN', 'Proc-Type', 'Version', 'DEK-Info']
```

--------------------------------

### Run All Tests

Source: https://credsweeper.readthedocs.io/en/latest/_sources/develop.rst.txt

Execute all tests using pytest. Ensure you have pytest installed.

```bash
python -m pytest -s tests/
```

--------------------------------

### CredSweeper JSON Output Example

Source: https://credsweeper.readthedocs.io/en/latest/_sources/guide.rst.txt

An example of the JSON output format from CredSweeper, detailing a detected password.

```json
[
    {
        "rule": "Password",
        "severity": "medium",
        "confidence": "moderate",
        "ml_probability": 0.7925280332565308,
        "line_data_list": [
            {
                "line": "password = 'cackle!'",
                "line_num": 1,
                "path": "test_samples.docker",
                "info": "FILE:test_samples.docker|TAR:blobs/sha256/82a4962c3cfebb62a42c2fd5c120ea0706a9ae66f52f71f957c052c873c60775|TAR:password.gradle|STRUCT|STRING:0|RAW",
                "variable": "password",
                "variable_start": 0,
                "variable_end": 8,
                "value": "cackle!",
                "value_start": 12,
                "value_end": 19,
                "entropy": 2.52164
            }
        ]
    }
]

```

--------------------------------

### Scan bytes with CredSweeper

Source: https://credsweeper.readthedocs.io/en/latest/_sources/guide.rst.txt

Minimal example for scanning bytes using the CredSweeper Python library. Ensure CredSweeper and ByteContentProvider are imported.

```python
from credsweeper import CredSweeper, ByteContentProvider


to_scan = b"line one\npassword='cackle!'"
cred_sweeper = CredSweeper()
provider = ByteContentProvider(to_scan)
results = cred_sweeper.file_scan(provider)
for r in results:
    print(r)
```

```text
rule: Password | severity: medium | confidence: moderate | ml_probability: 0.9857242107391357 | line_data_list: [line: 'password = "cackle!"' | line_num: 2 | path:  | value: 'cackle!' | entropy_validation: BASE64STDPAD_CHARS 2.120590 False]
```

--------------------------------

### CredSweeper Rule Configuration Example

Source: https://credsweeper.readthedocs.io/en/latest/_sources/overall_architecture.rst.txt

Example of a rule configuration in YAML format for CredSweeper. Specifies detection parameters like name, severity, confidence, type, and values.

```yaml
    - name: API
      severity: medium
      confidence: moderate
      type: keyword
      values:
        - api
      filter_type: GeneralKeyword
      use_ml: true
      min_line_len: 11
      required_substrings:
        - api
      target:
        - code
```

--------------------------------

### Scan a list of strings with CredSweeper

Source: https://credsweeper.readthedocs.io/en/latest/_sources/guide.rst.txt

Minimal example for scanning a list of strings using the CredSweeper Python library. Ensure CredSweeper and StringContentProvider are imported.

```python
from credsweeper import CredSweeper, StringContentProvider


to_scan = ["line one", "password='in_line_2'"]
cred_sweeper = CredSweeper()
provider = StringContentProvider(to_scan)
results = cred_sweeper.file_scan(provider)
for r in results:
    print(r)
```

```text
rule: Password | severity: medium | confidence: moderate | ml_probability: 0.9857242107391357 | line_data_list: [line: 'password = "cackle!"' | line_num: 1 | path:  | value: 'cackle!' | entropy_validation: BASE64STDPAD_CHARS 2.120590 False]
```

--------------------------------

### ML validation with CredSweeper

Source: https://credsweeper.readthedocs.io/en/latest/_sources/guide.rst.txt

Example demonstrating ML validation for detected credentials. You can adjust the threshold to control the sensitivity of the ML model. Note that some strings might not be reported due to failing the ML validation.

```python
from credsweeper import CredSweeper, StringContentProvider, MlValidator, ThresholdPreset


to_scan = ["line one", "password='cackle!'", "secret='template'"]
cred_sweeper = CredSweeper()
provider = StringContentProvider(to_scan)

# You can select lower or higher threshold to get more or less reports respectively
threshold = ThresholdPreset.medium
validator = MlValidator(threshold=threshold)

results = cred_sweeper.file_scan(provider)
for candidate in results:
    # For each results detected by a CredSweeper, you can validate them using MlValidator
    is_credential, with_probability = validator.validate(candidate)
    if is_credential:
        print(candidate)
```

```text
rule: Password | severity: medium | confidence: moderate | ml_probability: 0.9857242107391357 | line_data_list: [line: 'password = "cackle!"' | line_num: 2 | path:  | value: 'cackle!' | entropy_validation: BASE64STDPAD_CHARS 2.120590 False]
```

--------------------------------

### Prepare Dockerfile for Scanning

Source: https://credsweeper.readthedocs.io/en/latest/guide.html

Create a minimal Dockerfile to include test samples for scanning.

```dockerfile
FROM scratch
ADD tests/samples /
```

--------------------------------

### Display CredSweeper Help

Source: https://credsweeper.readthedocs.io/en/latest/_sources/guide.rst.txt

Run this command to see a list of all available arguments and options for the CredSweeper CLI.

```bash
python -m credsweeper --help
```

--------------------------------

### Create Dummy Candidate Instance

Source: https://credsweeper.readthedocs.io/en/latest/_modules/credsweeper/credentials/candidate.html

Creates a dummy instance of a candidate for use in searching files by extension. This is useful for testing or placeholder purposes.

```python
    @classmethod
    def get_dummy_candidate(cls, config: Config, file_path: str, file_type: str, info: str, rule_name: str):
        """Create dummy instance to use in searching file by extension"""
        return cls(  #
            line_data_list=[LineData(config, '', -1, 0, file_path, file_type, info, cls.DUMMY_PATTERN)],
            patterns=[cls.DUMMY_PATTERN],  #
            rule_name=rule_name,  #
            severity=Severity.INFO,  #
            config=config,  #
            confidence=Confidence.WEAK)
```

--------------------------------

### Get CLI Output with CredSweeper

Source: https://credsweeper.readthedocs.io/en/latest/_sources/guide.rst.txt

This command demonstrates how to get CredSweeper's output directly in the console for a specific file.

```bash
python -m credsweeper --path tests/samples/password.gradle

```

--------------------------------

### FilesProvider Initialization

Source: https://credsweeper.readthedocs.io/en/latest/_modules/credsweeper/file_handler/files_provider.html

Initializes the FilesProvider with a list of paths to scan and an optional flag to skip ignored files.

```APIDOC
## FilesProvider Initialization

### Description
Initializes the FilesProvider for files from 'paths'.

### Method
__init__

### Parameters
#### Path Parameters
- **paths** (Sequence[Union[str, Path, io.BytesIO, Tuple[Union[str, Path], io.BytesIO]]]) - Required - List of parent paths of files to scan OR tuple of path and io.BytesIO.
- **skip_ignored** (Optional[bool]) - Optional - Boolean variable, checking the directory to the list of ignored directories from the gitignore file.

### Request Example
```python
FilesProvider(paths=['/path/to/files', 'another/path'], skip_ignored=True)
FilesProvider(paths=[('/file.txt', io.BytesIO(b'content'))])
```
```

--------------------------------

### Example JSON Report Output

Source: https://credsweeper.readthedocs.io/en/latest/guide.html

Structure of the JSON report generated by CredSweeper.

```json
[
...
    {
        "rule": "Password",
        "severity": "medium",
        "confidence": "moderate",
        "ml_probability": 0.7925280332565308,
        "line_data_list": [
            {
                "line": "password = 'cackle!'",
                "line_num": 1,
                "path": "test_samples.docker",
                "info": "FILE:test_samples.docker|TAR:blobs/sha256/82a4962c3cfebb62a42c2fd5c120ea0706a9ae66f52f71f957c052c873c60775|TAR:password.gradle|STRUCT|STRING:0|RAW",
                "variable": "password",
                "variable_start": 0,
                "variable_end": 8,
                "value": "cackle!",
                "value_start": 12,
                "value_end": 19,
                "entropy": 2.52164
            }
        ]
    },
...
]
```

--------------------------------

### Config Class Initialization

Source: https://credsweeper.readthedocs.io/en/latest/_modules/credsweeper/config/config.html

Details the initialization of the Config class, which takes a dictionary of configuration settings.

```APIDOC
## Class: Config

### Description
Class that contains configurations that can be changed by the user.

### Method
__init__

### Parameters
#### Request Body
- **config** (Dict[str, Any]) - A dictionary containing all configuration settings.
  - **exclude.pattern** (List[str]) - List of regex patterns to exclude files.
  - **exclude.path** (List[str]) - List of paths to exclude.
  - **exclude.containers** (List[str]) - List of container names to exclude.
  - **exclude.documents** (List[str]) - List of document names to exclude.
  - **exclude.extension** (List[str]) - List of file extensions to exclude.
  - **exclude.lines** (List[str]) - Optional. Set of lines to exclude.
  - **exclude.values** (List[str]) - Optional. Set of values to exclude.
  - **source_ext** (List[str]) - List of source file extensions to scan.
  - **source_quote_ext** (List[str]) - List of source file extensions that contain quotes.
  - **find_by_ext_list** (List[str]) - List of file extensions to specifically search within.
  - **bruteforce_list** (List[str]) - List of strings to check for brute-force patterns.
  - **check_for_literals** (bool) - Whether to check for literal strings.
  - **use_filters** (bool) - Whether to use filters for scanning.
  - **line_data_output** (List[str]) - List of output formats for line data.
  - **candidate_output** (List[str]) - List of output formats for candidate findings.
  - **find_by_ext** (bool) - Whether to find files by extension.
  - **size_limit** (Optional[str]) - Maximum file size limit (e.g., '10MB').
  - **pedantic** (bool) - Whether to enable pedantic mode.
  - **depth** (int) - Maximum directory depth to scan.
  - **doc** (bool) - Whether to include documentation files in the scan.
  - **severity** (str) - The severity level for findings (e.g., 'INFO', 'LOW', 'MEDIUM', 'HIGH', 'CRITICAL').
  - **max_url_cred_value_length** (int) - Maximum length for URL credential values.
  - **max_password_value_length** (int) - Maximum length for password values.
  - **pattern_len** (Optional[int]) - Optional. Default pattern length.

### Request Example
```json
{
  "exclude": {
    "pattern": ["\.log$", ".*\.tmp$"],
    "path": ["/tmp/", "/var/log/"]
  },
  "source_ext": [".py", ".js", ".java"],
  "severity": "MEDIUM",
  "size_limit": "50MB",
  "check_for_literals": true,
  "use_filters": false,
  "line_data_output": ["json"],
  "candidate_output": ["text"],
  "find_by_ext": true,
  "depth": 5,
  "doc": false,
  "max_url_cred_value_length": 100,
  "max_password_value_length": 50
}
```

### Response
#### Success Response (200)
- **self.exclude_patterns** (List[re.Pattern]) - Compiled regex patterns for excluded files.
- **self.exclude_paths** (List[str]) - List of excluded paths.
- **self.exclude_containers** (List[str]) - List of excluded container names.
- **self.exclude_documents** (List[str]) - List of excluded document names.
- **self.exclude_extensions** (List[str]) - List of excluded file extensions.
- **self.exclude_lines** (Set[str]) - Set of excluded lines.
- **self.exclude_values** (Set[str]) - Set of excluded values.
- **self.source_extensions** (List[str]) - List of source file extensions.
- **self.source_quote_ext** (List[str]) - List of source file extensions with quotes.
- **self.find_by_ext_list** (List[str]) - List of file extensions to find by.
- **self.bruteforce_list** (List[str]) - List of strings for brute-force checking.
- **self.check_for_literals** (bool) - Flag for checking literals.
- **self.not_allowed_path_pattern** (re.Pattern) - Compiled regex pattern for disallowed paths.
- **self.use_filters** (bool) - Flag for using filters.
- **self.line_data_output** (List[str]) - Output formats for line data.
- **self.candidate_output** (List[str]) - Output formats for candidate findings.
- **self.find_by_ext** (bool) - Flag for finding by extension.
- **self.size_limit** (Optional[int]) - Maximum file size limit in bytes.
- **self.pedantic** (bool) - Flag for pedantic mode.
- **self.depth** (int) - Maximum scan depth.
- **self.doc** (bool) - Flag for including documentation.
- **self.severity** (Severity) - Severity level.
- **self.max_url_cred_value_length** (int) - Maximum URL credential value length.
- **self.max_password_value_length** (int) - Maximum password value length.
- **self.pattern_len** (int) - Default pattern length.

#### Response Example
```json
{
  "exclude_patterns": [".*\\.log$", ".*\\.tmp$"],
  "exclude_paths": ["/tmp/", "/var/log/"],
  "source_extensions": [".py", ".js", ".java"],
  "severity": "MEDIUM",
  "size_limit": 52428800,
  "check_for_literals": true,
  "use_filters": false,
  "line_data_output": ["json"],
  "candidate_output": ["text"],
  "find_by_ext": true,
  "depth": 5,
  "doc": false,
  "max_url_cred_value_length": 100,
  "max_password_value_length": 50
}
```
```

--------------------------------

### Define comment_starts tuple

Source: https://credsweeper.readthedocs.io/en/latest/credsweeper.credentials.html

Tuple containing common comment start sequences.

```python
comment_starts = ('//', '* ', '# ', '/*', '<!––', '%{', '%', '...', '(*', '--', '--[[', '#=')
```

--------------------------------

### Initialize ByteContentProvider

Source: https://credsweeper.readthedocs.io/en/latest/_modules/credsweeper/file_handler/byte_content_provider.html

Initializes ByteContentProvider with byte content, optional file path, file type, and info. The content is transformed into an array of lines split by the newline character.

```python
def __init__(
            self,  #
            content: bytes,  #
            file_path: Optional[str] = None,  #
            file_type: Optional[str] = None,  #
            info: Optional[str] = None) -> None:
        """
        Parameters:
            content: The bytes are transformed to an array of lines with split by new line character.

        """
        super().__init__(file_path=file_path, file_type=file_type, info=info)
        self.__data = content
        self.__lines: Optional[List[str]] = None
```

--------------------------------

### Get Scanner Type

Source: https://credsweeper.readthedocs.io/en/latest/_modules/credsweeper/scanner/scanner.html

Determines the appropriate scanner class based on the rule type.

```APIDOC
## GET /scanner

### Description
Retrieves the appropriate scanner class based on the provided rule type.

### Method
GET

### Endpoint
/scanner

### Parameters
#### Query Parameters
- **rule_type** (string) - Required - The type of rule to determine the scanner (e.g., PATTERN, KEYWORD, MULTI, PEM_KEY).

### Response
#### Success Response (200)
- **scanner_class** (Type) - The class of the scanner to be used.

#### Response Example
```json
{
  "scanner_class": "SinglePattern"
}
```
```

--------------------------------

### PngScanner.match

Source: https://credsweeper.readthedocs.io/en/latest/_modules/credsweeper/deep_scanner/png_scanner.html

Checks if the provided binary data starts with the standard PNG file signature.

```APIDOC
## Static Method: match

### Description
Determines if the input byte sequence is a valid PNG file by checking the magic number prefix.

### Parameters
- **data** (bytes) - Required - The raw file data to inspect.

### Response
- **boolean** - Returns True if the data starts with the PNG signature (\x89PNG\r\n\x1a\n), otherwise False.
```

--------------------------------

### Candidate Initialization

Source: https://credsweeper.readthedocs.io/en/latest/_modules/credsweeper/credentials/candidate.html

Initializes a Candidate object with detected credential information.

```APIDOC
## Candidate Initialization

### Description
Initializes a Candidate object with detected credential information.

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
None

### Request Example
None

### Response
#### Success Response (200)
None

#### Response Example
None
```

--------------------------------

### Build, Save, and Scan Docker Image

Source: https://credsweeper.readthedocs.io/en/latest/guide.html

Commands to build a Docker image, save it to a file, and run CredSweeper with deep scanning enabled.

```bash
docker build . --tag test_samples
docker save test_samples --output test_samples.docker
python -m credsweeper --path test_samples.docker --save-json output.json --depth 3
```

--------------------------------

### GET /analysis/target

Source: https://credsweeper.readthedocs.io/en/latest/_modules/credsweeper/file_handler/data_content_provider.html

Retrieves a generator for analysis targets based on a minimum line length requirement.

```APIDOC
## GET /analysis/target

### Description
Returns a generator of AnalysisTarget objects for scanning purposes.

### Method
GET

### Endpoint
/analysis/target

### Parameters
#### Query Parameters
- **min_len** (integer) - Required - The minimal line length to scan.

### Response
#### Success Response (200)
- **generator** (AnalysisTarget) - A generator yielding AnalysisTarget objects.

#### Error Response (501)
- **NotImplementedError** - This method is currently not implemented.
```

--------------------------------

### Initialize CredSweeper Class

Source: https://credsweeper.readthedocs.io/en/latest/_modules/credsweeper/app.html

The constructor accepts various configuration parameters for scanning, including rule paths, ML model settings, and output formats.

```python
class CredSweeper:
    """Advanced credential analyzer base class.

    Parameters:
        credential_manager: CredSweeper credential manager object
        scanner: CredSweeper scanner object
        pool_count: number of pools used to run multiprocessing scanning
        config: dictionary variable, stores analyzer features
        json_filename: string variable, credential candidates export filename

    """

    def __init__(self,
                 rule_path: Union[None, str, Path] = None,
                 config_path: Optional[str] = None,
                 json_filename: Union[None, str, Path] = None,
                 xlsx_filename: Union[None, str, Path] = None,
                 stdout: bool = False,
                 color: bool = False,
                 hashed: bool = False,
                 subtext: bool = False,
                 sort_output: bool = False,
                 use_filters: bool = True,
                 pool_count: int = 1,
                 ml_batch_size: Optional[int] = None,
                 ml_threshold: Union[int, float, ThresholdPreset] = ThresholdPreset.medium,
                 ml_config: Union[None, str, Path] = None,
                 ml_model: Union[None, str, Path] = None,
                 ml_providers: Optional[str] = None,
                 find_by_ext: bool = False,
                 pedantic: bool = False,
                 depth: int = 0,
                 doc: bool = False,
                 severity: Union[Severity, str] = Severity.INFO,
                 size_limit: Optional[str] = None,
                 exclude_lines: Optional[List[str]] = None,
                 exclude_values: Optional[List[str]] = None,
                 thrifty: bool = False,
                 log_level: Optional[str] = None) -> None:
        """Initialize Advanced credential scanner.

        Args:
            rule_path: optional str variable, path of rule config file
                validation was the grained candidate model on machine learning
            config_path: optional str variable, path of CredSweeper config file
                default built-in config is used if None
            json_filename: optional string variable, path to save result to json
            xlsx_filename: optional string variable, path to save result to xlsx
            stdout: print results to stdout
            color: print concise results to stdout with colorization
            hashed: use hash of line, value and variable instead plain text
            subtext: use subtext of line near variable-value like it performed in ML
            use_filters: boolean variable, specifying the need of rule filters
            pool_count: int value, number of parallel processes to use
            ml_batch_size: int value, size of the batch for model inference
            ml_threshold: float or string value to specify threshold for the ml model
            ml_config: str or Path to set custom config of ml model
            ml_model: str or Path to set custom ml model
            ml_providers: str - comma separated list with providers
            find_by_ext: boolean - files will be reported by extension
            pedantic: boolean - scan all files
            depth: int - how deep container files will be scanned
            doc: boolean - document-specific scanning
            severity: Severity - minimum severity level of rule
            size_limit: optional string integer or human-readable format to skip oversize files
            exclude_lines: lines to omit in scan. Will be added to the lines already in config
            exclude_values: values to omit in scan. Will be added to the values already in config
            thrifty: free provider resources after scan to reduce memory consumption
            log_level: str - level for pool initializer according logging levels (UPPERCASE)

        """
        self.pool_count: int = max(1, int(pool_count))
        if not (_severity := Severity.get(severity)):
            raise RuntimeError(f"Severity level provided: {severity}"
```

--------------------------------

### DiffContentProvider Initialization

Source: https://credsweeper.readthedocs.io/en/latest/_modules/credsweeper/file_handler/diff_content_provider.html

Initializes the DiffContentProvider with file path, change type, and diff data. The info parameter combines file path and change type for identification.

```python
class DiffContentProvider(ContentProvider):
    """Provide data from a single `.patch` file.

    Parameters:
        file_path: path to file
        change_type: set added or deleted file data to scan
        diff: list of file row changes, with base elements represented as::

            {
                "old": line number before diff,
                "new": line number after diff,
                "line": line text,
                "hunk": diff hunk number
            }

    """

    def __init__(
            self,  #
            file_path: str,  #
            change_type: DiffRowType,  #
            diff: List[DiffDict]) -> None:
        super().__init__(file_path=file_path, info=f"{file_path}:{change_type.value}")
        self.__change_type = change_type
        self.__diff = diff
```

--------------------------------

### Get Chunks

Source: https://credsweeper.readthedocs.io/en/latest/_modules/credsweeper/utils/util.html

Returns chunk positions for a given line length, useful for processing long lines.

```APIDOC
## get_chunks

### Description
Returns chunks positions for given line length.

### Method
staticmethod

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
- **line_len** (int) - Required - The total length of the line.

### Request Example
```json
{
  "line_len": 500
}
```

### Response
#### Success Response (200)
- **List[Tuple[int, int]]** - A list of tuples, where each tuple represents the start and end position of a chunk.

#### Response Example
```json
{
  "example": "[(0, 100), (100, 200), (200, 300)]"
}
```
```

--------------------------------

### Get File Extension

Source: https://credsweeper.readthedocs.io/en/latest/_modules/credsweeper/utils/util.html

Retrieves the file extension from a given file path. By default, the extension is returned in lowercase.

```python
import os

class Util:
    @staticmethod
    def get_extension(file_path: str, lower=True) -> str:
        """Return extension of file in lower case by default e.g.: '.txt', '.JPG'"""
        _, extension = os.path.splitext(str(file_path))
        return extension.lower() if lower else extension
```

--------------------------------

### JKS Scanner - Match Method

Source: https://credsweeper.readthedocs.io/en/latest/_modules/credsweeper/deep_scanner/jks_scanner.html

This method checks if a given byte array starts with the JKS file signature.

```APIDOC
## POST /websites/credsweeper_readthedocs_io_en

### Description
Checks if the provided data matches the JKS file signature.

### Method
POST

### Endpoint
/websites/credsweeper_readthedocs_io_en

### Parameters
#### Path Parameters
- **data** (bytes | bytearray) - Required - The byte data to check for the JKS signature.

### Request Example
```json
{
  "data": "\xfe\xed\xfe\xed..."
}
```

### Response
#### Success Response (200)
- **match** (boolean) - True if the data starts with the JKS signature, False otherwise.

#### Response Example
```json
{
  "match": true
}
```
```

--------------------------------

### Load and Merge Configuration Settings

Source: https://credsweeper.readthedocs.io/en/latest/_modules/credsweeper/app.html

Loads configuration from a JSON file and merges it with provided settings. Handles exclusion rules for lines and values.

```python
def _get_config_dict(
            self,  #
            config_path: Optional[str],  #
            use_filters: bool,  #
            find_by_ext: bool,  #
            pedantic: bool,  #
            depth: int,  #
            doc: bool,  #
            severity: Severity,  #
            size_limit: Optional[str],  #
            exclude_lines: Optional[List[str]],  #
            exclude_values: Optional[List[str]]) -> Dict[str, Any]:
        config_dict = Util.json_load(self._get_config_path(config_path))
        config_dict["use_filters"] = use_filters
        config_dict["find_by_ext"] = find_by_ext
        config_dict["size_limit"] = size_limit
        config_dict["pedantic"] = pedantic
        config_dict["depth"] = depth
        config_dict["doc"] = doc
        config_dict["severity"] = severity.value

        if exclude_lines is not None:
            config_dict["exclude"]["lines"] = config_dict["exclude"].get("lines", []) + exclude_lines
        if exclude_values is not None:
            config_dict["exclude"]["values"] = config_dict["exclude"].get("values", []) + exclude_values

        return config_dict  # type: ignore
```

--------------------------------

### Initialize Candidate Object

Source: https://credsweeper.readthedocs.io/en/latest/_modules/credsweeper/credentials/candidate.html

Initializes a Candidate object with line data, patterns, rule name, severity, configuration, ML usage flag, and confidence. ML probability is initialized to None.

```python
def __init__(self,
                 line_data_list: List[LineData],
                 patterns: List[re.Pattern],
                 rule_name: str,
                 severity: Severity,
                 config: Optional[Config] = None,
                 use_ml: bool = False,
                 confidence: Confidence = Confidence.MODERATE) -> None:
        self.line_data_list = line_data_list
        self.patterns = patterns
        self.rule_name = rule_name
        self.severity = severity
        self.config = config
        self.use_ml = use_ml
        self.confidence = confidence
        # None - ML is not applicable or not processed yet; float - the ml decision above ml_threshold
        # Note: -1.0 is possible too for some activation functions in ml model, so let avoid negative values
        self.ml_probability: Optional[float] = None
```

--------------------------------

### Media Format Detection Patterns

Source: https://credsweeper.readthedocs.io/en/latest/_modules/credsweeper/deep_scanner/deep_scanner.html

A dictionary mapping initial bytes to specific file format signatures and regex patterns for identification.

```python
    # manually crafted dict to detect a media format with first byte, prefix and optionally pattern
    MEDIA_PATTERNS: Dict[int, List[Tuple[bytes, re.Pattern]]] = {
        0x00: [
            (b"\x00\x00\x00\x0C\x6A\x50\x20\x20\x0D\x0A\x87\x0A", None),  # JPEG2000
            (b"\x00\x00\x01\x00", None),  # ICO
        ],
        0x1A: [
            (b"\x1A\x45\xDF\xA3", None),  # Matroska
        ],
        0x89: [
            (b"\x89PNG\x0D\x0A\x1A\x0A", None),  # PNG - can store text chunks inside
        ],
        0xFF: [
            (b"\xFF", re.compile(b"\xFF(\xD8\xFF[\xDB\xEE\xE1\xE0\x51]|[\xFB\xF3\xF2])")),  # JPEG or MPEG-1 Layer 3
        ],
        ord('B'): [
            (b"BM", re.compile(b"BM.{2}\x00{4}")),  # BMP
        ],
        ord('G'): [
            (b"GIF8", re.compile(b"GIF8[79]a.{0,4096}[\x00-\x08\x0C\x0E\x1F\x80-\xFF]")),  # GIF
        ],
        ord('I'): [
            (b"II", re.compile(b"II[+*]\x00.{0,4096}[\x00-\x08\x0C\x0E\x1F\x80-\xFF]")),  # TIFF little endian
        ],
        ord('M'): [
            (b"MM", re.compile(b"MM\x00[+*].{0,4096}[\x00-\x08\x0C\x0E\x1F\x80-\xFF]")),  # TIFF big endian
        ],
        ord('O'): [
            (b"OggS", re.compile(b"OggS.{0,4096}[\x00-\x08\x0C\x0E\x1F\x80-\xFF]")),  # OGG
        ],
        ord('R'): [
            (b"RIF", re.compile(b"RIF[FX].{4}[ 0-9A-Za-z]{4}.{0,4096}[\x00-\x08\x0C\x0E\x1F\x80-\xFF]")),  # RIFF va
        ],
        ord('X'): [
            (b"XFIR", re.compile(b"XFIR.{4}[ 0-9A-Za-z]{4}.{0,4096}[\x00-\x08\x0C\x0E\x1F\x80-\xFF]")),  # Macromedia
        ],
        ord('f'): [

```

--------------------------------

### Get group features

Source: https://credsweeper.readthedocs.io/en/latest/_modules/credsweeper/ml_model/ml_validator.html

Prepares inputs for model inference by encoding line, variable, and value data into batch-ready numpy arrays.

```python
    def get_group_features(self, candidates: List[Candidate]) -> Tuple[np.ndarray, np.ndarray, np.ndarray, np.ndarray]:
        """
        `np.newaxis` used to add new dimension if front, so input will be treated as a batch
        """
        # all candidates are from the same line
        default_candidate = candidates[0]
        line_input = self.encode_line(default_candidate.line_data_list[0].line,
                                      default_candidate.line_data_list[0].value_start)[np.newaxis]
        variable = ''
        value = ''
        for candidate in candidates:
            if not variable and candidate.line_data_list[0].variable:
                variable = candidate.line_data_list[0].variable
            if not value and candidate.line_data_list[0].value:
                value = candidate.line_data_list[0].value
            if variable and value:
                break
        variable_input = self.encode_value(variable)[np.newaxis]
        value_input = self.encode_value(value)[np.newaxis]
        feature_array = self.extract_features(candidates)
        return line_input, variable_input, value_input, feature_array
```

--------------------------------

### PEM Begin Regular Expression

Source: https://credsweeper.readthedocs.io/en/latest/credsweeper.utils.html

Regular expression to identify the start of a PEM private key block, excluding encrypted keys.

```python
re_pem_begin _ = re.compile('(?P<value>-----BEGIN\\\s(?!ENCRYPTED)[^-]*PRIVATE[^-]*KEY[^-]*-----(.+-----END[^-]+KEY[^-]*-----)?)')
```

--------------------------------

### Initialize Config Class in Python

Source: https://credsweeper.readthedocs.io/en/latest/_modules/credsweeper/config/config.html

Initializes the Config class with a dictionary of configuration settings. This class manages various settings like exclude patterns, source extensions, and severity levels. Ensure all required keys are present in the config dictionary.

```python
import re
from typing import Dict, List, Optional, Set, Any

from humanfriendly import parse_size

from credsweeper.common.constants import Severity, DEFAULT_PATTERN_LEN
from credsweeper.utils.util import Util




class Config:
    """Class that contain configs that can be changed by user."""

    NOT_ALLOWED_PATH = [
        ".*\.min\.js", ".*message.*\.properties", ".*locale.*\.properties", ".*makefile.*", ".*package-lock\.json",
        ".*package\.json", ".*\.css", ".*\.scss"
    ]

    def __init__(self, config: Dict[str, Any]) -> None:
        self.exclude_patterns: List[re.Pattern] = [re.compile(pattern) for pattern in config["exclude"]["pattern"]]
        self.exclude_paths: List[str] = config["exclude"]["path"]
        self.exclude_containers: List[str] = config["exclude"]["containers"]
        self.exclude_documents: List[str] = config["exclude"]["documents"]
        self.exclude_extensions: List[str] = config["exclude"]["extension"]
        self.exclude_lines: Set[str] = set(config["exclude"].get("lines", []))
        self.exclude_values: Set[str] = set(config["exclude"].get("values", []))
        self.source_extensions: List[str] = config["source_ext"]
        self.source_quote_ext: List[str] = config["source_quote_ext"]
        self.find_by_ext_list: List[str] = config["find_by_ext_list"]
        self.bruteforce_list: List[str] = config["bruteforce_list"]
        self.check_for_literals: bool = config["check_for_literals"]
        self.not_allowed_path_pattern = re.compile(f"{Util.get_regex_combine_or(self.NOT_ALLOWED_PATH)}",
                                                   flags=re.IGNORECASE)
        self.use_filters: bool = config["use_filters"]
        self.line_data_output: List[str] = config["line_data_output"]
        self.candidate_output: List[str] = config["candidate_output"]
        self.find_by_ext: bool = config["find_by_ext"]
        self.size_limit: Optional[int] = parse_size(config["size_limit"]) if config["size_limit"] is not None else None
        self.pedantic: bool = bool(config["pedantic"])
        self.depth: int = int(config["depth"])
        self.doc: bool = config["doc"]
        self.severity: Severity = Severity.get(config.get("severity"))

        self.max_url_cred_value_length: int = int(config["max_url_cred_value_length"])
        self.max_password_value_length: int = int(config["max_password_value_length"])

        # Trim exclude patterns from space like characters
        self.exclude_lines = set(line.strip() for line in self.exclude_lines)
        self.exclude_values = set(line.strip() for line in self.exclude_values)

        self.pattern_len = config.get("pattern_len", DEFAULT_PATTERN_LEN)

```

--------------------------------

### Get ML Validator Instance

Source: https://credsweeper.readthedocs.io/en/latest/_modules/credsweeper/app.html

Provides lazy initialization for the MlValidator. Creates an instance if it doesn't exist, using configured ML parameters.

```python
@property
def ml_validator(self) -> MlValidator:
        """ml_validator getter"""
        if not self.__ml_validator:
            self.__ml_validator = MlValidator(
                threshold=self.ml_threshold,  #
                ml_config=self.ml_config,  #
                ml_model=self.ml_model,  #
                ml_providers=self.ml_providers,  #
            )
        if not self.__ml_validator:
            raise RuntimeError("MlValidator was not initialized!")
        return self.__ml_validator
```

--------------------------------

### Initialize CredSweeper Configuration

Source: https://credsweeper.readthedocs.io/en/latest/_modules/credsweeper/app.html

Initializes the CredSweeper configuration with various parameters. Handles default configuration path and merges user-provided settings.

```python
def __init__(
        self,
        rule_path: str,
        config_path: Optional[str] = None,
        use_filters: bool = True,
        find_by_ext: bool = True,
        pedantic: bool = False,
        depth: int = 1,
        doc: bool = False,
        severity: Severity = Severity.INFO,
        size_limit: Optional[str] = None,
        exclude_lines: Optional[List[str]] = None,
        exclude_values: Optional[List[str]] = None,
        json_filename: Union[None, str, Path] = None,
        xlsx_filename: Union[None, str, Path] = None,
        stdout: bool = False,
        color: bool = False,
        hashed: bool = False,
        subtext: bool = False,
        sort_output: bool = False,
        ml_batch_size: int = 0,
        ml_threshold: int = 0,
        ml_config: Optional[str] = None,
        ml_model: Optional[str] = None,
        ml_providers: Optional[List[str]] = None,
        thrifty: bool = False,
        log_level: str = "INFO",
    ):
        self.config_path = config_path
        self.rule_path = rule_path
        self.use_filters = use_filters
        self.find_by_ext = find_by_ext
        self.pedantic = pedantic
        self.depth = depth
        self.doc = doc
        self.severity = severity
        self.size_limit = size_limit
        self.exclude_lines = exclude_lines
        self.exclude_values = exclude_values
        self.json_filename = json_filename
        self.xlsx_filename = xlsx_filename
        self.stdout = stdout
        self.color = color
        self.hashed = hashed
        self.subtext = subtext
        self.sort_output = sort_output
        self.ml_batch_size = ml_batch_size if ml_batch_size and 0 < ml_batch_size else 16
        self.ml_threshold = ml_threshold
        self.ml_config = ml_config
        self.ml_model = ml_model
        self.ml_providers = ml_providers
        self.__thrifty = thrifty
        self.__log_level = log_level
        self.__ml_validator: Optional[MlValidator] = None

                               f" -- must be one of: {' | '.join([i.value for i in Severity])}")
        config_dict = self._get_config_dict(config_path=config_path,
                                            use_filters=use_filters,
                                            find_by_ext=find_by_ext,
                                            pedantic=pedantic,
                                            depth=depth,
                                            doc=doc,
                                            severity=_severity,
                                            size_limit=size_limit,
                                            exclude_lines=exclude_lines,
                                            exclude_values=exclude_values)
        self.config = Config(config_dict)
        self.scanner = Scanner(self.config, rule_path)
        self.deep_scanner = DeepScanner(self.config, self.scanner)
        self.credential_manager = CredentialManager()
        self.json_filename: Union[None, str, Path] = json_filename
        self.xlsx_filename: Union[None, str, Path] = xlsx_filename
        self.stdout = stdout
        self.color = color
        self.hashed = hashed
        self.subtext = subtext
        self.sort_output = sort_output
        self.ml_batch_size = ml_batch_size if ml_batch_size and 0 < ml_batch_size else 16
        self.ml_threshold = ml_threshold
        self.ml_config = ml_config
        self.ml_model = ml_model
        self.ml_providers = ml_providers
        self.__thrifty = thrifty
        self.__log_level = log_level
        self.__ml_validator: Optional[MlValidator] = None
```

--------------------------------

### Get CredSweeper Configuration Path

Source: https://credsweeper.readthedocs.io/en/latest/_modules/credsweeper/app.html

Determines the configuration file path. Returns the provided path or the default config path if none is specified.

```python
@staticmethod
def _get_config_path(config_path: Optional[str]) -> Path:
    if config_path:
        return Path(config_path)
    else:
        return APP_PATH / "secret" / "config.json"
```

--------------------------------

### Run Basic CLI Scan

Source: https://credsweeper.readthedocs.io/en/latest/guide.html

Execute a standard scan on a specific file.

```bash
python -m credsweeper --path tests/samples/password.gradle
```

```text
rule: Password | severity: medium | confidence: moderate | ml_probability: 0.9149653911590576 | line_data_list: [path: tests/samples/password.gradle | line_num: 1 | value: 'cackle!' | line: 'password = "cackle!"']
```

--------------------------------

### Check if data is a DEB archive

Source: https://credsweeper.readthedocs.io/en/latest/_modules/credsweeper/deep_scanner/deb_scanner.html

Use this static method to determine if a given byte sequence starts with the magic bytes of a DEB archive.

```python
import logging
import struct
from abc import ABC
from typing import List, Optional, Generator, Tuple, Union

from credsweeper.common.constants import MIN_DATA_LEN, UTF_8
from credsweeper.credentials.candidate import Candidate
from credsweeper.deep_scanner.abstract_scanner import AbstractScanner
from credsweeper.file_handler.data_content_provider import DataContentProvider
from credsweeper.utils.util import Util

logger = logging.getLogger(__name__)




class DebScanner(AbstractScanner, ABC):
    """Implements deb (ar) scanning"""

    __header_size = 60




    @staticmethod
    def match(data: Union[bytes, bytearray]) -> bool:
        """According https://en.wikipedia.org/wiki/Deb_(file_format)"""
        if data.startswith(b"!<arch>\n"):
            return True
        return False

```

--------------------------------

### Exclude Outputs Using CredSweeper Config

Source: https://credsweeper.readthedocs.io/en/latest/_sources/guide.rst.txt

This example demonstrates how to configure CredSweeper to exclude specific lines or values by editing the 'exclude' section of its configuration file.

```json
"exclude": {
    "lines": ["   password = \"cackle!\" "],
    "values": ["cackle!"]
}

```

--------------------------------

### Initialize PatchesProvider

Source: https://credsweeper.readthedocs.io/en/latest/_modules/credsweeper/file_handler/patches_provider.html

Initializes the Files Patch Provider for patch files. Accepts a list of file paths and the type of change to analyze.

```python
def __init__(self, paths: Sequence[Union[str, Path, io.BytesIO, Tuple[Union[str, Path], io.BytesIO]]],
                 change_type: DiffRowType) -> None:
    """Initialize Files Patch Provider for patch files from 'paths'.

    Args:
        paths: file paths list to scan. All files should be in `.patch` format
        change_type: DiffRowType, type of analyses changes in patch (added or deleted)
          of ignored directories from the gitignore file

    """
    super().__init__(paths)
    self.change_type = change_type
```

--------------------------------

### Run CredSweeper from CLI

Source: https://credsweeper.readthedocs.io/en/latest/_sources/guide.rst.txt

Use the command-line interface to scan a specific file with a custom configuration.

```bash
$ python -m credsweeper --path tests/samples/password.gradle --config my_cfg.json
Detected Credentials: 0
Time Elapsed: 0.07152628898620605s
```

--------------------------------

### Define CandidateKey Class

Source: https://credsweeper.readthedocs.io/en/latest/_modules/credsweeper/credentials/candidate_key.html

Initializes a CandidateKey object with file path, line number, and value start/end positions. It generates a hashable key for comparison and equality checks.

```python
from typing import Tuple

from credsweeper.credentials.line_data import LineData




[docs]
class CandidateKey:
    """Class used to identify credential candidates.

    Candidates that detected same value on same string in a same file would have identical CandidateKey
    """

    def __init__(self, line_data: LineData):
        self.path: str = line_data.path
        self.line_num: int = line_data.line_num
        self.value_start: int = line_data.value_start
        self.value_end: int = line_data.value_end
        self.key: Tuple[str, int, int, int] = (self.path, self.line_num, self.value_start, self.value_end)
        self.__line = line_data.line

    def __hash__(self):
        return hash(self.key)

    def __eq__(self, other):
        return self.key == other.key

    def __ne__(self, other):
        return not bool(self == other)

    def __repr__(self) -> str:
        return f"{self.key}:{self.__line}"




```