Apache Flink

https://nightlies.apache.org/flink/flink-docs-release-1.14/
Admin
Apache Flink is a framework and distributed processing engine for stateful computations over...
Tokens:4,196,935
Snippets:61,111
Trust Score:10
Update:3 months ago
Show doc for...
Tokens:
Raw
### Run Gelly Examples

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/libs/gelly/overview

Commands to start the cluster and execute Gelly example drivers.

```bash
./bin/start-cluster.sh
./bin/flink run examples/gelly/flink-gelly-examples_*.jar
```

--------------------------------

### void install()

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/runtime/security/modules/SecurityModule.html

Installs the security module. Throws a SecurityInstallException if the installation fails.

```APIDOC
## void install()

### Description
Installs the security module.

### Throws
- **SecurityModule.SecurityInstallException** - if the security module couldn't be installed.
```

--------------------------------

### Start Server

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/runtime/query/KvStateServer.html

Initializes and starts the server instance.

```java
void start()
    throws Throwable
```

--------------------------------

### start()

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/test/util/TestProcessBuilder.html

Starts the process configured by the builder.

```APIDOC
### Method
`public TestProcessBuilder.TestProcess start()`

Starts the process and returns a TestProcess object containing the process and its output.
```

--------------------------------

### start()

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/runtime/minicluster/MiniCluster.html

Starts the mini cluster based on the configured properties.

```APIDOC
## Method: start()

### Description
Starts the mini cluster, based on the configured properties.
```

--------------------------------

### start()

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/runtime/query/KvStateServer.html

Starts the Queryable State Server.

```APIDOC
### Method
`void start()`

### Description
Starts the server.

### Throws
- **Throwable** - If an error occurs during startup.
```

--------------------------------

### Complete Kafka Source and Sink Example

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/dev/python/table/python_table_api_connectors

A full script demonstrating the setup of a Kafka source and sink within a PyFlink streaming environment.

```python
from pyflink.table import TableEnvironment, EnvironmentSettings

def log_processing():
    env_settings = EnvironmentSettings.in_streaming_mode()
    t_env = TableEnvironment.create(env_settings)
    # specify connector and format jars
    t_env.get_config().get_configuration().set_string("pipeline.jars", "file:///my/jar/path/connector.jar;file:///my/jar/path/json.jar")
    
    source_ddl = """
            CREATE TABLE source_table(
                a VARCHAR,
                b INT
            ) WITH (
              'connector' = 'kafka',
              'topic' = 'source_topic',
              'properties.bootstrap.servers' = 'kafka:9092',
              'properties.group.id' = 'test_3',
              'scan.startup.mode' = 'latest-offset',
              'format' = 'json'
            )
            """

    sink_ddl = """
            CREATE TABLE sink_table(
                a VARCHAR
            ) WITH (
              'connector' = 'kafka',
              'topic' = 'sink_topic',
              'properties.bootstrap.servers' = 'kafka:9092',
              'format' = 'json'
            )
            """

    t_env.execute_sql(source_ddl)
    t_env.execute_sql(sink_ddl)

    t_env.sql_query("SELECT a FROM source_table") \
        .execute_insert("sink_table").wait()


if __name__ == '__main__':
    log_processing()
```

--------------------------------

### setup

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/runtime/io/network/partition/consumer/InputGate.html

Performs setup operations for the gate.

```APIDOC
## void setup()

### Description
Sets up the gate. This may be a heavy-weight, blocking operation.

### Throws
- **IOException**
```

--------------------------------

### void setup()

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/runtime/io/network/api/writer/ResultPartitionWriter.html

Performs setup for the partition. This is a potentially heavy-weight, blocking operation compared to just creation.

```APIDOC
## void setup()

### Description
Setup partition, potentially heavy-weight, blocking operation comparing to just creation.

### Throws
- IOException
```

--------------------------------

### Start Process

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/test/util/TestProcessBuilder.html

Starts the configured process.

```java
public TestProcessBuilder.TestProcess start()
                                     throws IOException
```

--------------------------------

### install()

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/runtime/security/modules/ZooKeeperModule.html

Installs the ZooKeeper security module.

```APIDOC
### Method

`public void install()`

#### Throws
- **SecurityModule.SecurityInstallException** - Thrown if the security module could not be installed.
```

--------------------------------

### start()

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/runtime/zookeeper/ZooKeeperSharedValue.html

Starts the shared value instance.

```APIDOC
### Method
`void start() throws Exception`
```

--------------------------------

### Install Security Module Method

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/runtime/security/modules/SecurityModule.html

Method to install the security module. Throws SecurityInstallException if installation fails.

```java
void install()
      throws SecurityModule.SecurityInstallException
```

--------------------------------

### start()

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/runtime/resourcemanager/ResourceManagerService.html

Starts the ResourceManagerService.

```APIDOC
## void start()

### Description
Starts the service.

### Throws
- Exception - if the service cannot be started
```

--------------------------------

### XADataSource Configuration Examples

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/connectors/datastream/jdbc

Examples for initializing XADataSource objects for PostgreSQL, MySQL, and Oracle.

```java
PGXADataSource xaDataSource = new org.postgresql.xa.PGXADataSource();
xaDataSource.setUrl("jdbc:postgresql://localhost:5432/postgres");
xaDataSource.setUser(username);
xaDataSource.setPassword(password);
```

```java
MysqlXADataSource xaDataSource = new com.mysql.cj.jdbc.MysqlXADataSource();
xaDataSource.setUrl("jdbc:mysql://localhost:3306/");
xaDataSource.setUser(username);
xaDataSource.setPassword(password);
```

```java
OracleXADataSource xaDataSource = new oracle.jdbc.xa.OracleXADataSource();
xaDataSource.setURL("jdbc:oracle:oci8:@");
xaDataSource.setUser("scott");
xaDataSource.setPassword("tiger");
```

--------------------------------

### start()

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/runtime/io/network/ConnectionManager.html

Starts the internal components required for network connection and communication.

```APIDOC
## int start()

### Description
Starts the internal related components for network connection and communication.

### Returns
- **int** - A port to connect to the task executor for shuffle data exchange, or -1 if only local connection is possible.

### Throws
- **IOException**
```

--------------------------------

### start()

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/runtime/io/network/NettyShuffleEnvironment.html

Starts the network I/O components for the shuffle data exchange.

```APIDOC
## start()

### Description
Starts the network I/O components for the shuffle data exchange.

### Returns
- **int** - A port to connect for the shuffle data exchange, or -1 if only local connection is possible.

### Throws
- **IOException** - If an I/O error occurs during startup.
```

--------------------------------

### start()

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/kubernetes/highavailability/KubernetesCheckpointIDCounter.html

Starts the CheckpointIDCounter service.

```APIDOC
### Method
`public void start()`

Starts the CheckpointIDCounter service.
```

--------------------------------

### Get Installed Security Context

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/runtime/security/SecurityUtils.html

Retrieves the currently installed security context.

```java
public static SecurityContext getInstalledContext()
```

--------------------------------

### setup()

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/runtime/iterative/concurrent/SolutionSetUpdateBarrier.html

Initializes the barrier.

```APIDOC
## void setup()

### Description
Prepares the barrier for synchronization.
```

--------------------------------

### Get Consume Start Offset

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/connectors/hive/HiveTableSource.HiveContinuousPartitionFetcherContext.html

Retrieves the starting offset for partition consumption.

```java
public T getConsumeStartOffset()
```

--------------------------------

### start()

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/runtime/taskexecutor/TaskManagerRunner.TaskExecutorService.html

Starts the TaskExecutor service.

```APIDOC
## void start()

### Description
Starts the TaskExecutor service.

### Signature
`void start()`
```

--------------------------------

### Setup Method

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/runtime/operators/CoGroupWithSolutionSetFirstDriver.html

Initializes the driver with the provided task context.

```java
public void setup(TaskContext<CoGroupFunction<IT1,IT2,OT>,OT> context)
```

--------------------------------

### Get Checkpoint Start Delay Method

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/streaming/runtime/io/checkpointing/CheckpointBarrierHandler.html

Returns the delay encountered before the checkpoint started.

```java
public long getCheckpointStartDelayNanos()
```

--------------------------------

### start()

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/api/connector/source/SourceReader.html

Starts the source reader.

```APIDOC
### void start()

Starts the reader.
```

--------------------------------

### Example CLI Output

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/deployment/cli

Example output showing usage instructions and job submission confirmation.

```text
Usage with built-in data generator: StateMachineExample [--error-rate <probability-of-invalid-transition>] [--sleep <sleep-per-record-in-ms>]
Usage with Kafka: StateMachineExample --kafka-topic <topic> [--brokers <brokers>]
Options for both the above setups:
        [--backend <file|rocks>]
        [--checkpoint-dir <filepath>]
        [--async-checkpoints <true|false>]
        [--incremental-checkpoints <true|false>]
        [--output <filepath> OR null for stdout]

Using standalone source with error rate 0.000000 and sleep delay 1 millis

Job has been submitted with JobID 97b20a0a8ffd5c1d656328b0cd6436a6
```

--------------------------------

### Get Installed Security Modules

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/runtime/security/SecurityUtils.html

Retrieves the list of currently installed security modules.

```java
public static List<SecurityModule> getInstalledModules()
```

--------------------------------

### start

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/runtime/io/network/NettyShuffleEnvironment.html

Starts the internal services required by the shuffle environment.

```APIDOC
## start()

### Description
Start the internal related services before using the shuffle service environment.

### Returns
- **int** - Status code or identifier related to the start operation.

### Throws
- **IOException** - IO problem during startup.
```

--------------------------------

### Setup Method

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/runtime/operators/chaining/ChainedTerminationCriterionDriver.html

Initializes the driver with the parent invokable.

```java
public void setup(AbstractInvokable parent)
```

--------------------------------

### Get Async Checkpoint Start Delay

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/streaming/runtime/tasks/StreamTask.html

Returns the delay in nanoseconds before starting an asynchronous checkpoint.

```java
protected long getAsyncCheckpointStartDelayNanos()
```

--------------------------------

### Start Method

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/connector/kafka/source/enumerator/KafkaSourceEnumerator.html

Starts the enumerator and triggers partition discovery.

```java
public void start()
```

--------------------------------

### Install Security Module

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/runtime/security/modules/HadoopModule.html

Installs the security module, throwing a SecurityInstallException if the installation fails.

```java
public void install()
             throws SecurityModule.SecurityInstallException
```

--------------------------------

### Start Method

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/runtime/io/network/LocalConnectionManager.html

Starts the internal components for network connection and communication.

```java
public	int	start()
```

--------------------------------

### Display SQL Client Help

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/dev/table/sqlclient

Use this command to view all available CLI options and their descriptions.

```bash
./bin/sql-client.sh --help
```

--------------------------------

### Checkpoint Start Delay Methods

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/runtime/checkpoint/CheckpointMetricsBuilder.html

Methods to set and get the checkpoint start delay in nanoseconds.

```java
public CheckpointMetricsBuilder setCheckpointStartDelayNanos(long checkpointStartDelayNanos)
```

```java
public long getCheckpointStartDelayNanos()
```

--------------------------------

### start()

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/runtime/operators/coordination/OperatorCoordinator.html

Starts the coordinator. This method is called once at the beginning, before any other methods.

```APIDOC
## void start()

### Description
Starts the coordinator. This method is called once at the beginning, before any other methods.

### Throws
- **Exception** - Any exception thrown from this method causes a full job failure.
```

--------------------------------

### INSERT SQL Examples

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/dev/table/sql/insert

Various examples demonstrating how to append or overwrite data in partitioned tables using static and dynamic partitions.

```sql
-- Creates a partitioned table
CREATE TABLE country_page_view (user STRING, cnt INT, date STRING, country STRING)
PARTITIONED BY (date, country)
WITH (...)

-- Appends rows into the static partition (date='2019-8-30', country='China')
INSERT INTO country_page_view PARTITION (date='2019-8-30', country='China')
  SELECT user, cnt FROM page_view_source;

-- Appends rows into partition (date, country), where date is static partition with value '2019-8-30',
-- country is dynamic partition whose value is dynamic determined by each row.
INSERT INTO country_page_view PARTITION (date='2019-8-30')
  SELECT user, cnt, country FROM page_view_source;

-- Overwrites rows into static partition (date='2019-8-30', country='China')
INSERT OVERWRITE country_page_view PARTITION (date='2019-8-30', country='China')
  SELECT user, cnt FROM page_view_source;

-- Overwrites rows into partition (date, country), where date is static partition with value '2019-8-30',
-- country is dynamic partition whose value is dynamic determined by each row.
INSERT OVERWRITE country_page_view PARTITION (date='2019-8-30')
  SELECT user, cnt, country FROM page_view_source;

-- Appends rows into the static partition (date='2019-8-30', country='China')
-- the column cnt is set to NULL
INSERT INTO country_page_view PARTITION (date='2019-8-30', country='China') (user)
  SELECT user FROM page_view_source;
```

--------------------------------

### ResettableIterator newIterator()

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/table/runtime/util/ResettableRowBuffer.html

Get a new iterator starting from the first row.

```APIDOC
### ResettableIterator newIterator()

#### Description
Get a new iterator starting from first row.
```

--------------------------------

### start()

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/runtime/jobmaster/JobManagerRunner.html

Starts the execution of the JobMaster.

```APIDOC
void start() throws Exception
```

--------------------------------

### Initialize SQL Client with an initialization file

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/dev/table/sqlclient

Example of a SQL initialization file defining catalogs, databases, tables, views, functions, and execution properties.

```sql
-- Define available catalogs

CREATE CATALOG MyCatalog
  WITH (
    'type' = 'hive'
  );

USE CATALOG MyCatalog;

-- Define available database

CREATE DATABASE MyDatabase;

USE MyDatabase;

-- Define TABLE

CREATE TABLE MyTable(
  MyField1 INT,
  MyField2 STRING
) WITH (
  'connector' = 'filesystem',
  'path' = '/path/to/something',
  'format' = 'csv'
);

-- Define VIEW

CREATE VIEW MyCustomView AS SELECT MyField2 FROM MyTable;

-- Define user-defined functions here.

CREATE FUNCTION foo.bar.AggregateUDF AS myUDF;

-- Properties that change the fundamental execution behavior of a table program.

SET 'execution.runtime-mode' = 'streaming'; -- execution mode either 'batch' or 'streaming'
SET 'sql-client.execution.result-mode' = 'table'; -- available values: 'table', 'changelog' and 'tableau'
SET 'sql-client.execution.max-table-result.rows' = '10000'; -- optional: maximum number of maintained rows
SET 'parallelism.default' = '1'; -- optional: Flink's parallelism (1 by default)
SET 'pipeline.auto-watermark-interval' = '200'; --optional: interval for periodic watermarks
SET 'pipeline.max-parallelism' = '10'; -- optional: Flink's maximum parallelism
SET 'table.exec.state.ttl' = '1000'; -- optional: table program's idle state time
SET 'restart-strategy' = 'fixed-delay';

-- Configuration options for adjusting and tuning table programs.

SET 'table.optimizer.join-reorder-enabled' = 'true';
SET 'table.exec.spill-compression.enabled' = 'true';
SET 'table.exec.spill-compression.block-size' = '128kb';
```

--------------------------------

### Get Range Boundaries

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/runtime/state/changelog/inmemory/InMemoryChangelogStateHandle.html

Retrieves the start and end boundaries of the key-group range.

```java
public long getFrom()
```

```java
public long getTo()
```

--------------------------------

### start()

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/table/client/gateway/local/LocalExecutor.html

Starts the executor and ensures it is ready for command execution.

```APIDOC
### Method
void start()

### Description
Starts the executor and ensures that it is ready for commands to be executed.
```

--------------------------------

### start()

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/api/connector/source/SplitEnumerator.html

Starts the split enumerator.

```APIDOC
## void start()

### Description
Starts the split enumerator. The default behavior does nothing.
```

--------------------------------

### Get First Record Start

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/api/common/io/BlockInfo.html

Retrieves the offset of the first record in the block.

```java
public long getFirstRecordStart()
```

--------------------------------

### void startUp()

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/connectors/test/common/TestResource.html

Starts up the test resource. This method should be implemented to be idempotent.

```APIDOC
### void startUp()

#### Description
Starts up the test resource. The implementation of this method should be idempotent.

#### Throws
- **Exception** - if anything goes wrong when starting the resource
```

--------------------------------

### External Resource Configuration Example

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/configuration/ExternalResourceOptions.html

Example configuration showing how to define multiple external resources and their associated parameters.

```text
 external-resources = gpu;fpga

 external-resource.gpu.driver-factory.class: org.apache.flink.externalresource.gpu.GPUDriverFactory
 external-resource.gpu.amount: 2
 external-resource.gpu.param.type: nvidia

 external-resource.fpga.driver-factory.class: org.apache.flink.externalresource.fpga.FPGADriverFactory
 external-resource.fpga.amount: 1
```

--------------------------------

### Get Superstep Number

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/api/common/functions/IterationRuntimeContext.html

Retrieves the current superstep number, starting from 1.

```java
int getSuperstepNumber()
```

--------------------------------

### start()

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/runtime/operators/sort/StageRunner.html

Starts the execution of the sorting stage.

```APIDOC
## void start()

### Description
Starts the sorting stage. This method is used to initiate the processing logic defined within the stage.

### Signature
`void start()`
```

--------------------------------

### install(SecurityConfiguration config)

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/runtime/security/SecurityUtils.html

Installs a process-wide security configuration.

```APIDOC
### Method
`public static void install(SecurityConfiguration config) throws Exception`

### Description
Applies a process-wide security configuration using the available security modules.

### Parameters
- **config** (SecurityConfiguration) - Required - The security configuration to be installed.

### Throws
- **Exception** - If the installation process fails.
```

--------------------------------

### Get Starting Position

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/streaming/connectors/kinesis/util/AWSUtil.html

Creates a StartingPosition object from sequence number and configuration properties.

```java
public static StartingPosition getStartingPosition(SequenceNumber sequenceNumber,
                                                   Properties configProps)
```

--------------------------------

### Gate Setup and Lifecycle

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/runtime/io/network/partition/consumer/SingleInputGate.html

Methods for initializing the gate, managing state, and handling partition requests.

```java
public void setup()
           throws IOException
```

```java
public CompletableFuture<Void> getStateConsumedFuture()
```

```java
public void requestPartitions()
```

```java
public void finishReadRecoveredState()
                              throws IOException
```

```java
public void close()
           throws IOException
```

```java
public boolean isFinished()
```

--------------------------------

### Get prefixed key-value pairs

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/configuration/ConfigurationUtils.html

Extracts configuration properties that start with a specific prefix.

```java
public static Map<String,String> getPrefixedKeyValuePairs(String prefix,
                                                          Configuration configuration)
```

--------------------------------

### ResettableIterator newIterator(int beginRow)

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/table/runtime/util/ResettableRowBuffer.html

Get a new iterator starting from the specified beginRow index.

```APIDOC
### ResettableIterator newIterator(int beginRow)

#### Description
Get a new iterator starting from the beginRow-th row. beginRow is 0-indexed.

#### Parameters
- **beginRow** (int) - The 0-indexed starting position for the iterator.
```

--------------------------------

### startUp()

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/connectors/test/common/environment/MiniClusterTestEnvironment.html

Starts up the test resource.

```APIDOC
## void startUp()

### Description
Starts up the test resource. This method is idempotent.
```

--------------------------------

### startUp()

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/connectors/test/common/environment/RemoteClusterTestEnvironment.html

Starts up the test resource.

```APIDOC
## Method: startUp()

### Description
Start up the test resource. The implementation of this method should be idempotent.
```

--------------------------------

### Get Start Command from Template

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/runtime/clusterframework/BootstrapTools.html

Replaces placeholders in a command template with values from a provided map.

```java
public static String getStartCommand(String template,
                                     Map<String,String> startCommandValues)
```

--------------------------------

### Get Start Command with Bash Wrapper

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/kubernetes/utils/KubernetesUtils.html

Wraps a command string with a bash execution context.

```java
public static List<String> getStartCommandWithBashWrapper(String command)
```

--------------------------------

### Setup Partition

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/runtime/io/network/api/writer/ResultPartitionWriter.html

Performs initialization for the partition, which may be a heavy-weight or blocking operation.

```java
void setup()
    throws IOException
```

--------------------------------

### Get Default Edge DataSet

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/graph/examples/data/ConnectedComponentsDefaultData.html

Retrieves the default edge dataset for the connected components example.

```java
public static DataSet<Edge<Long,NullValue>> getDefaultEdgeDataSet(ExecutionEnvironment env)
```

--------------------------------

### Start a local Flink cluster

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/dev/table/sql/gettingstarted

Execute this command from the Flink installation folder to initialize a local cluster.

```bash
./bin/start-cluster.sh
```

--------------------------------

### start(JobLeaderIdActions initialJobLeaderIdActions)

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/runtime/resourcemanager/JobLeaderIdService.html

Starts the service with the provided job leader actions.

```APIDOC
## void start(JobLeaderIdActions initialJobLeaderIdActions)

### Description
Starts the service with the given job leader actions.

### Parameters
- **initialJobLeaderIdActions** (JobLeaderIdActions) - Required - The actions to use for job leader id operations.

### Throws
- **Exception** - Thrown when clearing up old state.
```

--------------------------------

### start(JobLeaderIdActions initialJobLeaderIdActions)

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/runtime/resourcemanager/DefaultJobLeaderIdService.html

Starts the service with the provided job leader actions.

```APIDOC
### Method
`void start(JobLeaderIdActions initialJobLeaderIdActions)`

### Description
Starts the service with the given job leader actions.

### Parameters
- **initialJobLeaderIdActions** (JobLeaderIdActions) - Required - The actions to use for job leader ID operations.

### Throws
- **Exception** - Thrown when clearing up old state.
```

--------------------------------

### Get Head Element Index Method

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/runtime/state/heap/HeapPriorityQueue.html

Returns the starting index of the queue elements in the internal array.

```java
protected int getHeadElementIndex()
```

--------------------------------

### setup

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/streaming/api/operators/async/AsyncWaitOperator.html

Initializes the operator by setting access to the context and the output.

```APIDOC
## Method: setup

### Description
Initializes the operator. Sets access to the context and the output.

### Signature
public void setup(StreamTask<?,?> containingTask, StreamConfig config, Output<StreamRecord<OUT>> output)
```

--------------------------------

### Get Initial Solution Set

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/api/java/operators/DeltaIteration.html

Retrieves the initial solution set used at the start of the delta iteration.

```java
public	DataSet<ST>	getInitialSolutionSet()
```

--------------------------------

### Run WordCount Example

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/dev/dataset/examples

Commands to execute the WordCount example jar with default or custom input/output paths.

```bash
./bin/flink run ./examples/batch/WordCount.jar
```

```bash
./bin/flink run ./examples/batch/WordCount.jar --input /path/to/some/text/data --output /path/to/result
```

--------------------------------

### Start Server

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/queryablestate/network/AbstractServerBase.html

Starts the server by binding to the configured address. This operation is blocking.

```java
public void start()
           throws Throwable
```

--------------------------------

### Get Key From Args

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/api/java/Utils.html

Parses a key from an argument array, where keys are expected to start with '-' or '--'.

```java
public static String getKeyFromArgs(String[] args,
                                    int index)
```

--------------------------------

### Define Pattern Quantifiers in Scala

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/libs/cep

Examples of applying various quantifiers to a pattern named 'start' in Scala.

```scala
// expecting 4 occurrences
start.times(4)

// expecting 0 or 4 occurrences
start.times(4).optional()

// expecting 2, 3 or 4 occurrences
start.times(2, 4)

// expecting 2, 3 or 4 occurrences and repeating as many as possible
start.times(2, 4).greedy()

// expecting 0, 2, 3 or 4 occurrences
start.times(2, 4).optional()

// expecting 0, 2, 3 or 4 occurrences and repeating as many as possible
start.times(2, 4).optional().greedy()

// expecting 1 or more occurrences
start.oneOrMore()

// expecting 1 or more occurrences and repeating as many as possible
start.oneOrMore().greedy()

// expecting 0 or more occurrences
start.oneOrMore().optional()

// expecting 0 or more occurrences and repeating as many as possible
start.oneOrMore().optional().greedy()

// expecting 2 or more occurrences
start.timesOrMore(2)

// expecting 2 or more occurrences and repeating as many as possible
start.timesOrMore(2).greedy()

// expecting 0, 2 or more occurrences
start.timesOrMore(2).optional()

// expecting 0, 2 or more occurrences and repeating as many as possible
start.timesOrMore(2).optional().greedy()
```

--------------------------------

### RMQSource Connection and Queue Setup Methods

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/streaming/connectors/rabbitmq/RMQSource.html

Methods for initializing connections and configuring queues, which can be overridden for custom implementations.

```java
protected com.rabbitmq.client.ConnectionFactory setupConnectionFactory()
                                                                throws Exception
```

```java
@VisibleForTesting
protected com.rabbitmq.client.Connection setupConnection()
                                                                     throws Exception
```

```java
@VisibleForTesting
protected void setupQueue()
                                      throws IOException
```

--------------------------------

### Start Bundle

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/streaming/api/runners/python/beam/BeamPythonFunctionRunner.html

Internal method used for testing to start a bundle.

```java
@VisibleForTesting
protected void startBundle()
```

--------------------------------

### startInstance(String portsConfig)

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/management/jmx/JMXService.html

Starts the JVM-wide singleton JMX server.

```APIDOC
### Method
`static void startInstance(String portsConfig)`

### Description
Starts the JVM-wide singleton JMX server. If the server is already running, it will not be restarted, and a warning will be logged.

### Parameters
- **portsConfig** (String) - The port configuration for the JMX server.
```

--------------------------------

### Define Pattern Quantifiers in Java

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/libs/cep

Examples of applying various quantifiers to a pattern named 'start' in Java.

```java
// expecting 4 occurrences
start.times(4);

// expecting 0 or 4 occurrences
start.times(4).optional();

// expecting 2, 3 or 4 occurrences
start.times(2, 4);

// expecting 2, 3 or 4 occurrences and repeating as many as possible
start.times(2, 4).greedy();

// expecting 0, 2, 3 or 4 occurrences
start.times(2, 4).optional();

// expecting 0, 2, 3 or 4 occurrences and repeating as many as possible
start.times(2, 4).optional().greedy();

// expecting 1 or more occurrences
start.oneOrMore();

// expecting 1 or more occurrences and repeating as many as possible
start.oneOrMore().greedy();

// expecting 0 or more occurrences
start.oneOrMore().optional();

// expecting 0 or more occurrences and repeating as many as possible
start.oneOrMore().optional().greedy();

// expecting 2 or more occurrences
start.timesOrMore(2);

// expecting 2 or more occurrences and repeating as many as possible
start.timesOrMore(2).greedy();

// expecting 0, 2 or more occurrences
start.timesOrMore(2).optional()

// expecting 0, 2 or more occurrences and repeating as many as possible
start.timesOrMore(2).optional().greedy();
```

--------------------------------

### Execute Quickstart Script

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/dev/datastream/project-configuration

Runs the Flink quickstart script via bash to initialize a project.

```bash
bash -c "$(curl https://flink.apache.org/q/gradle-quickstart.sh)" -- 1.14.4 _2.11
```

--------------------------------

### Accessors and Mutators for WindowFrameSpec

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/table/planner/delegation/hive/copy/HiveParserWindowingSpec.WindowFrameSpec.html

Methods to get and set the start and end boundaries, retrieve the window type, and convert the object to a string.

```java
public HiveParserWindowingSpec.BoundarySpec getStart()
public void setStart(HiveParserWindowingSpec.BoundarySpec start)
public HiveParserWindowingSpec.BoundarySpec getEnd()
public void setEnd(HiveParserWindowingSpec.BoundarySpec end)
public HiveParserWindowingSpec.WindowType getWindowType()
public String toString()
```

--------------------------------

### public static void main(String[] args)

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/streaming/kafka/test/KafkaExample.html

The main entry point for the Kafka example application. It accepts command-line arguments to configure Kafka topics, bootstrap servers, and consumer group IDs.

```APIDOC
## public static void main(String[] args)

### Description
Executes the Kafka example job. This method reads from an input Kafka topic, processes events, and writes to an output Kafka topic.

### Parameters
- **args** (String[]) - Command-line arguments including --input-topic, --output-topic, --bootstrap.servers, and --group.id.

### Throws
- **Exception** - Thrown if the execution fails.
```

--------------------------------

### Remote Debugging Setup for Python UDFs

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/dev/python/debugging

Install the required tool and inject the trace command into your UDF to enable remote debugging.

```bash
$ pip install pydevd-pycharm
```

```python
import pydevd_pycharm
pydevd_pycharm.settrace('localhost', port=6789, stdoutToServer=True, stderrToServer=True)
```

--------------------------------

### Environment Setup Methods

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/test/util/MultipleProgramsTestBase.html

Methods for setting up and tearing down the test environment.

```java
public void setupEnvironment()
```

```java
public void teardownEnvironment()
```

--------------------------------

### Example Configuration

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/deployment/ha/kubernetes_ha

A complete example of the required configuration settings for flink-conf.yaml.

```yaml
kubernetes.cluster-id: <cluster-id>
high-availability: org.apache.flink.kubernetes.highavailability.KubernetesHaServicesFactory
high-availability.storageDir: hdfs:///flink/recovery
```

--------------------------------

### Start Up Test Resource

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/connectors/test/common/environment/MiniClusterTestEnvironment.html

Starts the test resource. This implementation should be idempotent.

```java
public void startUp()
             throws Exception
```

--------------------------------

### start()

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/runtime/shuffle/ShuffleEnvironment.html

Initializes the shuffle service environment. This method must be called before using the shuffle service.

```APIDOC
## void start()

### Description
Initializes the shuffle service environment. This must be called before any shuffle operations are performed.
```

--------------------------------

### createReader(SourceReaderContext readerContext)

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/api/connector/source/lib/NumberSequenceSource.html

Creates a new reader to read data from the splits it gets assigned. The reader starts fresh and does not have any state to resume.

```APIDOC
## createReader

### Description
Creates a new reader to read data from the splits it gets assigned. The reader starts fresh and does not have any state to resume.

### Parameters
- **readerContext** (SourceReaderContext) - The context for the source reader.

### Returns
- **SourceReader** - A new SourceReader.
```

--------------------------------

### public static void main(String[] args)

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/streaming/kinesis/test/KinesisExampleTest.html

The main entry point for the KinesisExampleTest driver.

```APIDOC
## public static void main(String[] args)

### Description
Executes the test driver for KinesisExample.main.

### Parameters
- **args** (String[]) - Command line arguments passed to the test driver.

### Throws
- **Exception** - Thrown if an error occurs during execution.
```

--------------------------------

### StreamPartitioner setup Method

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/streaming/runtime/partitioner/StreamPartitioner.html

Initializes the partitioner with the specified number of output channels.

```java
public void setup(int numberOfChannels)
```

--------------------------------

### Start JMX Server

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/management/jmx/JMXService.html

Start the JVM-wide singleton JMX server. If already started, a warning is logged.

```java
public static void startInstance(String portsConfig)
```

--------------------------------

### public static void main(String[] args)

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/streaming/examples/statemachine/StateMachineExample.html

The main entry point for the StateMachineExample application. This method initializes and executes the streaming job.

```APIDOC
## public static void main(String[] args)

### Description
Main entry point for the StateMachineExample program. This method starts the streaming application that evaluates state machine rules for incoming event streams.

### Parameters
- **args** (String[]) - Required - The command line arguments for the application.

### Throws
- **Exception** - Thrown if the application fails to execute.
```

--------------------------------

### Building Configuration Options

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/python/_modules/pyflink/common/config_options.html

Examples of defining configuration options with different types and default value settings.

```python
# simple string-valued option with a default value
>>> ConfigOptions.key("tmp.dir").string_type().default_value("/tmp")
# simple integer-valued option with a default value
>>> ConfigOptions.key("application.parallelism").int_type().default_value(100)
# option with no default value
>>> ConfigOptions.key("user.name").string_type().no_default_value()
```

--------------------------------

### public static void main(String[] args)

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/examples/java/relational/util/WebLogDataGenerator.html

Generates sample data files for the WebLogAnalysis example program. The method creates three files in the system temporary directory: documents, ranks, and visits.

```APIDOC
## public static void main(String[] args)

### Description
Generates sample data for the WebLogAnalysis example. It creates files for web documents, ranks, and visits in the system temporary directory.

### Parameters
- **args** (String[]) - Required - An array containing two integers: 1. Number of web documents, 2. Number of visits.
```

--------------------------------

### setup(int numberOfChannels)

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/runtime/io/network/api/writer/ChannelSelector.html

Initializes the channel selector with the total number of output channels attached to the output gate.

```APIDOC
### void setup(int numberOfChannels)

#### Description
Initializes the channel selector with the number of output channels.

#### Parameters
- **numberOfChannels** (int) - The total number of output channels which are attached to the respective output gate.
```

--------------------------------

### start(String, RpcService, HighAvailabilityServices, JobLeaderListener)

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/runtime/taskexecutor/JobLeaderService.html

Starts the job leader service with the provided services.

```APIDOC
## void start(String initialOwnerAddress, RpcService initialRpcService, HighAvailabilityServices initialHighAvailabilityServices, JobLeaderListener initialJobLeaderListener)

### Description
Starts the job leader service with the given services.

### Parameters
- **initialOwnerAddress** (String) - The address to be used for establishing connections.
- **initialRpcService** (RpcService) - The service used to create RPC connections.
- **initialHighAvailabilityServices** (HighAvailabilityServices) - The services used to create leader retrieval services for jobs.
- **initialJobLeaderListener** (JobLeaderListener) - The listener for job leader changes.
```

--------------------------------

### public static void main(String[] args)

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/client/python/PythonGatewayServer.html

Starts a local GatewayServer on an ephemeral port. This method communicates the port information to the Python side via a file.

```APIDOC
## public static void main(String[] args)

### Description
Starts a local GatewayServer on an ephemeral port. This method is the entry point for the gateway and communicates the server details to the Python process via a file.

### Signature
`public static void main(String[] args) throws IOException, ExecutionException, InterruptedException`

### Parameters
- **args** (String[]) - Command line arguments passed to the server.

### Throws
- **IOException** - If an I/O error occurs.
- **ExecutionException** - If an execution error occurs.
- **InterruptedException** - If the thread is interrupted.
```

--------------------------------

### prepare(TemporaryFolder tempFolder, String... additionalPrincipals)

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/test/util/SecureTestEnvironment.html

Prepares the secure test environment by initializing the MiniKDC.

```APIDOC
## prepare(TemporaryFolder tempFolder, String... additionalPrincipals)

### Description
Initializes the MiniKDC environment for testing purposes.

### Parameters
- **tempFolder** (org.junit.rules.TemporaryFolder) - Required - The temporary folder used for KDC data.
- **additionalPrincipals** (String...) - Optional - Additional principals to be added to the KDC.
```

--------------------------------

### start Method

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/table/api/internal/BaseExpressions.html

Returns the start time (inclusive) of a window when applied on a window reference.

```java
public OutType start()
```

--------------------------------

### start(LeaderContender newContender)

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/runtime/leaderelection/StandaloneLeaderElectionService.html

Starts the leader election service with the provided contender.

```APIDOC
## start(LeaderContender newContender)

### Description
Starts the leader election service. This method can only be called once.

### Parameters
- **newContender** (LeaderContender) - Required - The LeaderContender which applies for the leadership.
```

--------------------------------

### Implement startUp Method

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/connectors/test/common/TestResource.html

Starts the test resource. Implementations must be idempotent.

```java
void	startUp()
      throws Exception
```

--------------------------------

### Installing Security Configuration

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/test/util/TestingSecurityContext.html

Installs the security configuration and client security map for the test environment.

```java
public static
void
install(SecurityConfiguration config,
                           Map<String,org.apache.flink.test.util.TestingSecurityContext.ClientSecurityConfiguration> clientSecurityConfigurationMap)
                    throws Exception
```

--------------------------------

### createAndStart

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/runtime/rpc/RpcSystem.RpcServiceBuilder.html

Creates and starts the RpcService.

```APIDOC
### Method
`RpcService createAndStart() throws Exception`

### Description
Finalizes the configuration and initializes the RpcService, returning the started instance.
```

--------------------------------

### Open Initialization Method

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/table/examples/java/connectors/ChangelogCsvDeserializer.html

Performs one-time setup work before deserialization begins, using the provided InitializationContext.

```java
public	void	open(DeserializationSchema.InitializationContext	context)
```

--------------------------------

### Working with Table Objects

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/table/api/Table.html

Examples demonstrating how to create a table, perform transformations, and execute the pipeline.

```java

 TableEnvironment tableEnv = TableEnvironment.create(...);

 Table table = tableEnv.from("MyTable").select($("colA").trim(), $("colB").plus(12));

 table.execute().print();
 
```

```scala

 val tableEnv = TableEnvironment.create(...)

 val table = tableEnv.from("MyTable").select($"colA".trim(), $"colB" + 12)

 table.execute().print()
 
```

--------------------------------

### Install a Pluggable File System

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/deployment/filesystems/plugins

Commands to create a plugin directory and copy the required JAR file from the opt directory.

```bash
mkdir ./plugins/s3-fs-hadoop
cp ./opt/flink-s3-fs-hadoop-1.14.4.jar ./plugins/s3-fs-hadoop/
```

--------------------------------

### start Method

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/runtime/jobmanager/NoOpJobGraphStoreWatcher.html

Starts the watcher on JobGraphStore.

```java
public void start(JobGraphStore.JobGraphListener jobGraphListener)
```

--------------------------------

### get()

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/runtime/checkpoint/StandaloneCheckpointIDCounter.html

Gets the current checkpoint ID.

```APIDOC
## Method: get()

### Description
Atomically retrieves the current checkpoint ID.

### Returns
- **long** - The current checkpoint ID.
```

--------------------------------

### Streaming and Batch SQL with Filesystem Connector

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/connectors/table/filesystem

Demonstrates creating a source Kafka table and a partitioned filesystem sink table, followed by streaming insertion and batch selection queries.

```sql
CREATE TABLE kafka_table (
  user_id STRING,
  order_amount DOUBLE,
  log_ts TIMESTAMP(3),
  WATERMARK FOR log_ts AS log_ts - INTERVAL '5' SECOND
) WITH (...);

CREATE TABLE fs_table (
  user_id STRING,
  order_amount DOUBLE,
  dt STRING,
  `hour` STRING
) PARTITIONED BY (dt, `hour`) WITH (
  'connector'='filesystem',
  'path'='...',
  'format'='parquet',
  'sink.partition-commit.delay'='1 h',
  'sink.partition-commit.policy.kind'='success-file'
);

-- streaming sql, insert into file system table
INSERT INTO fs_table 
SELECT 
    user_id, 
    order_amount, 
    DATE_FORMAT(log_ts, 'yyyy-MM-dd'),
    DATE_FORMAT(log_ts, 'HH') 
FROM kafka_table;

-- batch sql, select with partition pruning
SELECT * FROM fs_table WHERE dt='2020-05-20' and `hour`='12';
```

--------------------------------

### start()

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/runtime/rpc/StartStoppable.html

Starts the processing of remote procedure calls.

```APIDOC
### void start()

Starts the processing of remote procedure calls.
```

--------------------------------

### Setup BufferWritingResultPartition

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/runtime/io/network/partition/BufferWritingResultPartition.html

Initializes the partition by registering the buffer pool.

```java
public void setup()
           throws IOException
```

--------------------------------

### Write Map Start

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/formats/avro/utils/DataOutputEncoder.html

Signals the start of a map.

```java
public void writeMapStart()
```

--------------------------------

### setup(StreamTask<?,?> containingTask, StreamConfig config, Output<StreamRecord<OUT>> output)

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/streaming/api/operators/AbstractStreamOperator.html

Initializes the operator by setting access to the context and the output.

```APIDOC
## setup

### Description
Initializes the operator. Sets access to the context and the output.

### Signature
`public void setup(StreamTask<?,?> containingTask, StreamConfig config, Output<StreamRecord<OUT>> output)`
```

--------------------------------

### get()

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/kubernetes/highavailability/KubernetesCheckpointIDCounter.html

Atomically gets the current checkpoint ID.

```APIDOC
### Method
`public long get()`

Returns the current checkpoint ID.
```

--------------------------------

### Start Client Method

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/table/client/SqlClient.html

Protected method to initialize the client, intended for testing purposes.

```java
@VisibleForTesting
protected static void startClient(String[] args,
                                                     java.util.function.Supplier<org.jline.terminal.Terminal> terminalFactory)
```

--------------------------------

### void before()

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/util/ExternalResource.html

Performs setup operations before a test is executed.

```APIDOC
### void before()

#### Description
Performs setup operations before a test is executed. This method may throw an Exception.

#### Throws
- **Exception** - If an error occurs during the setup process.
```

--------------------------------

### Pre-Execution Setup

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/python/_modules/pyflink/table/table_environment.html

Configures JARs and classpaths before job execution.

```python
    def _before_execute(self):
        jvm = get_gateway().jvm
        jars_key = jvm.org.apache.flink.configuration.PipelineOptions.JARS.key()
        classpaths_key = jvm.org.apache.flink.configuration.PipelineOptions.CLASSPATHS.key()
        self._add_jars_to_j_env_config(jars_key)
        self._add_jars_to_j_env_config(classpaths_key)
```

--------------------------------

### Start Operator Coordinators

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/runtime/scheduler/DefaultOperatorCoordinatorHandler.html

Starts all registered operator coordinators.

```java
public void startAllOperatorCoordinators()
```

--------------------------------

### SQL Query Examples

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/calcite/rel/logical/LogicalTableScan.html

Examples of SQL queries demonstrating table access constraints.

```sql
select from fields
```

```sql
select from fields as f
 where f.getClass().getName().equals("java.lang.String")
```

--------------------------------

### Constructor

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/yarn/configuration/YarnResourceManagerDriverConfiguration.html

Initializes the configuration with environment variables, RPC address, and web interface URL.

```java
public YarnResourceManagerDriverConfiguration(Map<String,String> env,
                                              String rpcAddress,
                                              @Nullable
                                              String webInterfaceUrl)
```

--------------------------------

### Run Cluster Entrypoint

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/runtime/entrypoint/ClusterEntrypoint.html

Starts the cluster entrypoint.

```java
public static void runClusterEntrypoint(ClusterEntrypoint clusterEntrypoint)
```

--------------------------------

### Start Method

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/runtime/jobmanager/ZooKeeperJobGraphStoreWatcher.html

Starts the watcher on the JobGraphStore and registers a listener.

```java
public
void
start(JobGraphStore.JobGraphListener
jobGraphListener)
           throws Exception
```

--------------------------------

### Initialize AbstractFileStateBackend with Path and Configuration

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/runtime/state/filesystem/AbstractFileStateBackend.html

Creates a backend instance using Path objects and a ReadableConfig for additional settings.

```java
protected AbstractFileStateBackend(@Nullable
                                   Path baseCheckpointPath,
                                   @Nullable
                                   Path baseSavepointPath,
                                   ReadableConfig configuration)
```

--------------------------------

### NoOpDriver Setup Method

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/runtime/operators/NoOpDriver.html

Initializes the driver with the provided TaskContext.

```java
public
void
setup(TaskContext<AbstractRichFunction,T> context)
```

--------------------------------

### Start Item

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/formats/avro/utils/DataOutputEncoder.html

Signals the start of an item in an array or map.

```java
public void startItem()
```

--------------------------------

### Full Iteration Example in Java

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/dev/datastream/overview

A complete example that subtracts 1 from integers until they reach zero.

```java
DataStream<Long> someIntegers = env.generateSequence(0, 1000);

IterativeStream<Long> iteration = someIntegers.iterate();

DataStream<Long> minusOne = iteration.map(new MapFunction<Long, Long>() {
  @Override
  public Long map(Long value) throws Exception {
    return value - 1 ;
  }
});

DataStream<Long> stillGreaterThanZero = minusOne.filter(new FilterFunction<Long>() {
  @Override
  public boolean filter(Long value) throws Exception {
    return (value > 0);
  }
});

iteration.closeWith(stillGreaterThanZero);

DataStream<Long> lessThanZero = minusOne.filter(new FilterFunction<Long>() {
  @Override
  public boolean filter(Long value) throws Exception {
    return (value <= 0);
  }
});
```

--------------------------------

### Set Start Cursor

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/connector/pulsar/source/PulsarSourceBuilder.html

Specifies the starting offset for the PulsarSource.

```java
public PulsarSourceBuilder<OUT> setStartCursor(StartCursor startCursor)
```

--------------------------------

### ChannelSelector Setup Method

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/runtime/io/network/api/writer/ChannelSelector.html

Initializes the selector with the total number of available output channels.

```java
void setup(int numberOfChannels)
```

--------------------------------

### Start the Docker environment

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/try-flink/table_api

Build and launch the containerized environment for the Flink table walkthrough.

```bash
$ docker-compose build
$ docker-compose up -d
```

--------------------------------

### Get position

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/table/planner/delegation/hive/copy/HiveParserRowResolver.html

Gets the position of a column by its internal name.

```java
public int getPosition(String internalName)
```

--------------------------------

### Basic HybridSource Configuration

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/connector/base/source/hybrid/HybridSource.html

Demonstrates creating a HybridSource that transitions from a FileSource to a KafkaSource with a fixed starting position.

```java
 FileSource<String> fileSource =
   FileSource.forRecordStreamFormat(new TextLineFormat(), Path.fromLocalFile(testDir)).build();
 KafkaSource<String> kafkaSource =
           KafkaSource.<String>builder()
                   .setBootstrapServers("localhost:9092")
                   .setGroupId("MyGroup")
                   .setTopics(Arrays.asList("quickstart-events"))
                   .setDeserializer(
                           KafkaRecordDeserializer.valueOnly(StringDeserializer.class))
                   .setStartingOffsets(OffsetsInitializer.earliest())
                   .build();
 HybridSource<String> hybridSource =
           HybridSource.builder(fileSource)
                   .addSource(kafkaSource)
                   .build();
```

--------------------------------

### Get innerPayLoad field

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/streaming/tests/avro/ComplexPayloadAvro.Builder.html

Gets the value of the 'innerPayLoad' field.

```java
public InnerPayLoadAvro getInnerPayLoad()
```

--------------------------------

### Configure Method

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/api/common/io/OutputFormat.html

Initializes the output format using the provided configuration parameters. This is the first method called after instantiation.

```java
void configure(Configuration parameters)
```

--------------------------------

### Get Schema Builder

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/fnexecution/v1/FlinkFnApi.CoderInfoDescriptor.OverWindowArrowType.Builder.html

Gets the builder for the schema field.

```java
public FlinkFnApi.Schema.Builder getSchemaBuilder()
```

--------------------------------

### public static void main(String[] args)

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/examples/java/clustering/util/KMeansDataGenerator.html

Generates data points and cluster centers for the KMeans example program. The output is saved to the specified path or a default temporary directory.

```APIDOC
## public static void main(String[] args)

### Description
Main method to generate data for the KMeans example program. It creates two files: one for data points and one for cluster centers.

### Parameters
- **args** (String[]) - Required - Array of arguments:
  1. Int: Number of data points
  2. Int: Number of cluster centers
  3. String (Optional): Output path (default: {tmp.dir})
  4. Double (Optional): Standard deviation of data points
  5. Double (Optional): Value range of cluster centers
  6. Long (Optional): Random seed

### Throws
- **IOException** - If an error occurs during file generation.
```

--------------------------------

### Mark Alignment Start

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/streaming/runtime/io/checkpointing/CheckpointBarrierHandler.html

Marks the start of the alignment process for a checkpoint.

```java
protected void markAlignmentStart(long checkpointId,
                                  long checkpointCreationTimestamp)
```

--------------------------------

### Initialization Method

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/formats/avro/AvroRowDataSerializationSchema.html

Method to perform one-time setup work before serialization.

```java
public void open(SerializationSchema.InitializationContext context)
          throws Exception
```

--------------------------------

### runFetcher()

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/streaming/connectors/kinesis/internals/KinesisDataFetcher.html

Starts the fetcher. After starting the fetcher, it can only be stopped by calling shutdownFetcher().

```APIDOC
## public void runFetcher()

### Description
Starts the fetcher. After starting the fetcher, it can only be stopped by calling shutdownFetcher().

### Throws
- **Exception** - the first error or exception thrown by the fetcher or any of the threads created by the fetcher.
```

--------------------------------

### Start TaskManager

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/deployment/resource-providers/standalone/overview

Launch a TaskManager to provide resources for the application.

```bash
$ ./bin/taskmanager.sh start
```

--------------------------------

### RMQSource Open Method

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/streaming/connectors/rabbitmq/RMQSource.html

Initializes the source with the provided configuration.

```java
public void open(Configuration config)
          throws Exception
```

--------------------------------

### Signal idling start

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/runtime/metrics/groups/InternalSourceReaderMetricGroup.html

Indicates that the source reader has started idling.

```java
public void idlingStarted()
```

--------------------------------

### Start Service

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/runtime/leaderelection/DefaultLeaderElectionService.html

Starts the leader election service. This method can only be called once.

```java
public final void start(LeaderContender contender)
                 throws Exception
```

--------------------------------

### CrossDriver Setup Method

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/runtime/operators/CrossDriver.html

Initializes the driver with the provided task context.

```java
public void setup(TaskContext<CrossFunction<T1,T2,OT>,OT> context)
```

--------------------------------

### Top-N Query Example

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/dev/table/sql/queries/topn

A practical example demonstrating how to retrieve the top five products per category based on sales.

```sql
CREATE TABLE ShopSales (
  product_id   STRING,
  category     STRING,
  product_name STRING,
  sales        BIGINT
) WITH (...);

SELECT *
FROM (
  SELECT *,
    ROW_NUMBER() OVER (PARTITION BY category ORDER BY sales DESC) AS row_num
  FROM ShopSales)
WHERE row_num <= 5
```

--------------------------------

### Start Method Definition

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/runtime/jobmanager/JobGraphStoreWatcher.html

Starts the watcher on the JobGraphStore with the provided listener.

```java
void start(JobGraphStore.JobGraphListener jobGraphListener)
    throws Exception
```

--------------------------------

### Initialize RocksDBStateBackend

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/python/_modules/pyflink/datastream/state_backend.html

Instantiate the RocksDBStateBackend with a URI pointing to the checkpoint storage location.

```python
>>> state_backend = RocksDBStateBackend("file://var/checkpoints/")
```

--------------------------------

### Handle checkpoint start

Source: https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/runtime/io/network/partition/consumer/RemoteInputChannel.html

Spills queued buffers when a checkpoint starts.

```java
public void checkpointStarted(CheckpointBarrier barrier)
                       throws CheckpointException
```
Tokens:
Raw