### Initialize Flink Quickstart Environment Source: https://github.com/apache/iceberg/blob/main/site/docs/flink-quickstart.md Clone the repository and start the required Docker containers for the Flink and Iceberg environment. ```sh git clone https://github.com/apache/iceberg.git cd iceberg docker compose -f docker/iceberg-flink-quickstart/docker-compose.yml up -d --build ``` -------------------------------- ### Install Daft and PyIceberg Source: https://github.com/apache/iceberg/blob/main/site/docs/integrations/daft.md Install the required libraries in your Python environment. ```bash pip install daft pyiceberg ``` -------------------------------- ### Install and Run Linkchecker Source: https://github.com/apache/iceberg/blob/main/site/README.md Commands to install the linkchecker tool and run it against a local development server. It also shows how to output the results to a CSV file for review. ```bash pip install linkchecker ./linkchecker http://localhost:8000 -r1 -Fcsv/link_warnings.csv cat ./link_warnings.csv ``` -------------------------------- ### Start Docker containers Source: https://github.com/apache/iceberg/blob/main/site/docs/spark-quickstart.md Execute this command to initialize the services defined in the docker-compose.yml file. ```sh docker-compose up ``` -------------------------------- ### Snapshot Log Example Source: https://github.com/apache/iceberg/blob/main/format/spec.md Provides an example of the snapshot log, which records historical snapshots with their IDs and timestamps. ```json [ { "snapshot-id": 3051729675574597004, "timestamp-ms": 1515100955770 } ] ``` -------------------------------- ### Table Properties Example Source: https://github.com/apache/iceberg/blob/main/format/spec.md Illustrates the structure for table properties, which are key-value pairs used for configuration. ```json { "write.format.default": "avro", "commit.retry.num-retries": "4" } ``` -------------------------------- ### SQL View Representation Examples Source: https://github.com/apache/iceberg/blob/main/format/view-spec.md Examples of SQL statements used to define or interact with views in Iceberg. ```sql USE prod.default ``` ```sql CREATE OR REPLACE VIEW event_agg ( event_count COMMENT 'Count of events', event_date) AS SELECT COUNT(1), CAST(event_ts AS DATE) FROM events GROUP BY 2 ``` -------------------------------- ### Configure RCK via Environment Variables Source: https://github.com/apache/iceberg/blob/main/open-api/README.md Example of configuring the catalog client using environment variables with the CATALOG_ prefix. ```shell CATALOG_URI=https://my_rest_server.io/ ## -> uri=https://my_rest_server.io/ CATALOG_WAREHOUSE=test_warehouse ## -> warehouse=test_warehouse CATALOG_IO__IMPL=org.apache.iceberg.aws.s3.S3FileIO ## -> io-impl=org.apache.iceberg.aws.s3.S3FileIO CATALOG_CREDENTIAL=: ## -> credential=: ``` -------------------------------- ### Example of system.bucket() Function Source: https://github.com/apache/iceberg/blob/main/docs/docs/spark-queries.md Illustrates the usage of the system.bucket() function, which returns an integer representing the bucket for a given column value and number of buckets. ```sql SELECT system.bucket(16, id) FROM prod.db.table; ``` -------------------------------- ### Snapshot Log Entry Example Source: https://github.com/apache/iceberg/blob/main/site/docs/spec.md Provides an example of a single entry in the snapshot log, containing snapshot ID and timestamp. ```json [ { "snapshot-id": 3051729675574597004, "timestamp-ms": 1515100955770 } ] ``` -------------------------------- ### Configure RCK via Java System Properties Source: https://github.com/apache/iceberg/blob/main/open-api/README.md Example of configuring the catalog client using Java system properties with the rck. prefix. ```shell rck.uri=https://my_rest_server.io/ ## -> uri=https://my_rest_server.io/ rck.warehouse=test_warehouse ## -> warehouse=test_warehouse rck.io-impl=org.apache.iceberg.aws.s3.S3FileIO ## -> io-impl=org.apache.iceberg.aws.s3.S3FileIO rck.credential=: ## -> credential=: ``` -------------------------------- ### SQL Representation Metadata Example Source: https://github.com/apache/iceberg/blob/main/site/docs/view-spec.md Example of metadata for a SQL representation, showing the type, SQL query, and dialect. ```text | Field name | Value | |---------------------|-------| | `type` | "sql" | | `sql` | "SELECT\n COUNT(1), CAST(event_ts AS DATE)\nFROM events\nGROUP BY 2" | | `dialect` | "spark" | ``` -------------------------------- ### Metadata Log Entry Example Source: https://github.com/apache/iceberg/blob/main/site/docs/spec.md An example of an entry in the metadata log, referencing a metadata file and its timestamp. ```json [ { "metadata-file": "s3://bucket/.../v1.json", "timestamp-ms": 1515100955770 } ] ``` -------------------------------- ### Metadata Log Example Source: https://github.com/apache/iceberg/blob/main/format/spec.md Demonstrates the structure of the metadata log, which tracks metadata file changes and their timestamps. ```json [ { "metadata-file": "s3://bucket/.../v1.json", "timestamp-ms": 1515100955770 } ] ``` -------------------------------- ### Start Spark Shell for Iceberg Verification Source: https://github.com/apache/iceberg/blob/main/site/docs/how-to-release.md Use this command to start a spark-shell configured with the necessary Iceberg runtime and catalog settings for verification. ```bash spark-shell \ --conf spark.jars.repositories=${MAVEN_URL} \ --packages org.apache.iceberg:iceberg-spark-runtime-3.5_2.12:{{ icebergVersion }} \ --conf spark.sql.extensions=org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions \ --conf spark.sql.catalog.local=org.apache.iceberg.spark.SparkCatalog \ --conf spark.sql.catalog.local.type=hadoop \ --conf spark.sql.catalog.local.warehouse=$PWD/warehouse \ --conf spark.sql.catalog.local.default-namespace=default \ --conf spark.sql.defaultCatalog=local ``` -------------------------------- ### Initialize Flink SQL Client with SQL File Source: https://github.com/apache/iceberg/blob/main/docs/docs/flink-ddl.md Execute an initialization SQL file when starting the Flink SQL Client using the `-i` option. ```bash /path/to/bin/sql-client.sh -i /path/to/init.sql ``` -------------------------------- ### Build Iceberg Flink Quickstart Docker Image Locally Source: https://github.com/apache/iceberg/blob/main/docker/iceberg-flink-quickstart/README.md Build the Docker image locally using default versions. This command creates an image tagged as apache/iceberg-flink-quickstart. ```bash docker build -t apache/iceberg-flink-quickstart docker/iceberg-flink-quickstart/ ``` -------------------------------- ### Build Iceberg Project Locally Source: https://github.com/apache/iceberg/blob/main/site/docs/contribute.md Use Gradle to build the Iceberg project and run tests. Ensure you have Java 17 or 21 installed. ```bash ./gradlew build ``` ```bash ./gradlew build -x test -x integrationTest ``` ```bash ./gradlew spotlessApply ``` ```bash ./gradlew build -DsparkVersions=3.5,4.0 -DflinkVersions=1.20,2.0 ``` -------------------------------- ### Table Reference Example Source: https://github.com/apache/iceberg/blob/main/format/spec.md Illustrates the structure for table references, used for tags or other named snapshots. ```json { "test": { "snapshot-id": 123456789000, "type": "tag", "max-ref-age-ms": 10000000 } } ``` -------------------------------- ### Start Spark Session Source: https://github.com/apache/iceberg/blob/main/examples/Convert table to Iceberg.ipynb Initiates a Spark session. Ensure the iceberg-runtime JAR is added to your Spark environment. ```python spark ``` -------------------------------- ### Partition Field JSON Example Source: https://github.com/apache/iceberg/blob/main/site/docs/spec.md An example of a single partition field within a partition spec. It shows the source ID, field ID, name, and the transform applied. ```json { "source-id": 4, "field-id": 1000, "name": "ts_day", "transform": "day" } ``` -------------------------------- ### Equality Delete File Example 2 Source: https://github.com/apache/iceberg/blob/main/format/spec.md An example of an equality delete file that includes the full row data for the deleted row. This is an alternative way to represent the delete for 'id = 3'. ```text equality_ids=[1] 1: id | 2: category | 3: name -------|-------------|--------- 3 | NULL | Grizzly ``` -------------------------------- ### Equality Delete File Example 3 Source: https://github.com/apache/iceberg/blob/main/format/spec.md An example of an equality delete file used to delete rows where 'id' equals 4 AND 'category' is NULL. This demonstrates using multiple columns for equality matching. ```text equality_ids=[1, 2] 1: id | 2: category | 3: name -------|-------------|--------- 4 | NULL | Polar ``` -------------------------------- ### Execute Flink Streaming Read Source: https://github.com/apache/iceberg/blob/main/docs/docs/flink-queries.md Configure streaming mode and use SQL hints to define monitoring intervals and snapshot start points. ```sql -- Submit the flink job in streaming mode for current session. SET execution.runtime-mode = streaming; -- Enable this switch because streaming read SQL will provide few job options in flink SQL hint options. SET table.dynamic-table-options.enabled=true; -- Read all the records from the iceberg current snapshot, and then read incremental data starting from that snapshot. SELECT * FROM sample /*+ OPTIONS('streaming'='true', 'monitor-interval'='1s')*/ ; -- Read all incremental data starting from the snapshot-id '3821550127947089987' (records from this snapshot will be excluded). SELECT * FROM sample /*+ OPTIONS('streaming'='true', 'monitor-interval'='1s', 'start-snapshot-id'='3821550127947089987')*/ ; ``` -------------------------------- ### Sort Field JSON Example Source: https://github.com/apache/iceberg/blob/main/site/docs/spec.md An example of a single sort field within a sort order. It specifies the transform, source ID, sorting direction, and how null values should be ordered. ```json { "transform": "identity", "source-id": 2, "direction": "asc", "null-order": "nulls-first" } ``` -------------------------------- ### Sort Order JSON Example Source: https://github.com/apache/iceberg/blob/main/site/docs/spec.md This is an example of a sort order serialized as a list of JSON objects. Each object defines a sort field with transform, source ID, direction, and null order. ```json [ { "transform": "identity", "source-id": 2, "direction": "asc", "null-order": "nulls-first" }, { "transform": "bucket[4]", "source-id": 3, "direction": "desc", "null-order": "nulls-last" } ] ``` -------------------------------- ### Equality Delete File Example 1 Source: https://github.com/apache/iceberg/blob/main/format/spec.md An example of an equality delete file used to delete rows where the 'id' column equals 3. The 'equality_ids' metadata column specifies which table columns are used for matching. ```text equality_ids=[1] 1: id ------- 3 ``` -------------------------------- ### Encryption Key Example Source: https://github.com/apache/iceberg/blob/main/format/spec.md Shows the format for encryption key information within the table metadata. ```json [ {"key-id": "5f819b", "key-metadata": "aWNlYmVyZwo="} ] ``` -------------------------------- ### Create a Partition Spec Source: https://github.com/apache/iceberg/blob/main/docs/docs/java-api-quickstart.md Build a PartitionSpec for a table's schema to define how records are grouped into data files. This example partitions by hour and identity. ```java import org.apache.iceberg.PartitionSpec; PartitionSpec spec = PartitionSpec.builderFor(schema) .hour("event_time") .identity("level") .build(); ``` -------------------------------- ### List Tables in a Catalog with pyiceberg Source: https://github.com/apache/iceberg/blob/main/docker/iceberg-rest-fixture/README.md To list tables within a specific catalog, provide the catalog name after the `list` command. This example lists tables in the 'nyc' catalog. ```bash ➜ ~ pyiceberg --uri http://localhost:8181 list nyc nyc.taxis nyc.taxis3 nyc.taxis4 nyc.taxis_copy_maybe yc.taxis_skeleton yc.taxis_skeleton2 ``` -------------------------------- ### Snapshot Object Example Source: https://github.com/apache/iceberg/blob/main/site/docs/spec.md Shows the structure of a single snapshot object within the snapshots list in table metadata. ```json [ { "snapshot-id": 3051729675574597004, "timestamp-ms": 1515100955770, "summary": { "operation": "append" }, "manifest-list": "s3://b/wh/.../s1.avro" "schema-id": 0 } ] ``` -------------------------------- ### SQL to Create Iceberg Destination Table Source: https://github.com/apache/iceberg/blob/main/docs/docs/kafka-connect.md Example SQL statement to create a destination table in Iceberg, partitioned by hours of a timestamp column. ```sql CREATE TABLE default.events ( id STRING, type STRING, ts TIMESTAMP, payload STRING) PARTITIONED BY (hours(ts)) ``` -------------------------------- ### Initialize Table and Data Source: https://github.com/apache/iceberg/blob/main/docs/docs/branching.md Create a sample table and insert initial records. ```sql CREATE TABLE db.table (id bigint, data string, col float); INSERT INTO db.table VALUES (1, 'a', 1.0), (2, 'b', 2.0), (3, 'c', 3.0); SELECT * FROM db.table; 1 a 1.0 2 b 2.0 3 c 3.0 ``` -------------------------------- ### Name Mapping Serialization Example Source: https://github.com/apache/iceberg/blob/main/format/spec.md This JSON structure defines how field names are mapped, including nested fields and alternative names. ```json [ { "field-id": 1, "names": ["id", "record_id"] }, { "field-id": 2, "names": ["data"] }, { "field-id": 3, "names": ["location"], "fields": [ { "field-id": 4, "names": ["latitude", "lat"] }, { "field-id": 5, "names": ["longitude", "long"] } ] } ] ``` -------------------------------- ### Configure URL Connection Socket Timeout Source: https://github.com/apache/iceberg/blob/main/docs/docs/aws.md Example of configuring the socket timeout for the URL Connection HTTP Client when starting a Spark shell. This sets the timeout to 80 milliseconds. ```shell --conf spark.sql.catalog.my_catalog.http-client.urlconnection.socket-timeout-ms=80 ``` -------------------------------- ### Quick Start: Table Maintenance Job Source: https://github.com/apache/iceberg/blob/main/docs/docs/flink-maintenance.md A comprehensive example demonstrating how to set up an automated table maintenance job in Flink, including options for external lock factories or Flink-managed locks. ```APIDOC ### Quick Start The following example demonstrates the implementation of automated maintenance for an Iceberg table within a Flink environment. ```java StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment(); TableLoader tableLoader = TableLoader.fromCatalog( CatalogLoader.hive("my_catalog", configuration, properties), TableIdentifier.of("database", "table") ); Map jdbcProps = new HashMap<>(); jdbcProps.put("jdbc.user", "flink"); jdbcProps.put("jdbc.password", "flinkpw"); //JdbcLockFactory Example TriggerLockFactory lockFactory = new JdbcLockFactory( "jdbc:postgresql://localhost:5432/iceberg", // JDBC URL "catalog.db.table", // Lock ID (unique identifier) jdbcProps // JDBC connection properties ); // Option 1: With external lock factory (plan to deprecate this Option since 1.12) TableMaintenance.forTable(env, tableLoader, lockFactory) // Option 2: With Flink-managed lock (no external lock required) TableMaintenance.forTable(env, tableLoader) .uidSuffix("my-maintenance-job") .rateLimit(Duration.ofMinutes(10)) .lockCheckDelay(Duration.ofSeconds(10)) .add(ExpireSnapshots.builder() .scheduleOnCommitCount(10) .maxSnapshotAge(Duration.ofMinutes(10)) .retainLast(5) .deleteBatchSize(5) .parallelism(8)) .add(RewriteDataFiles.builder() .scheduleOnDataFileCount(10) .targetFileSizeBytes(128 * 1024 * 1024) .partialProgressEnabled(true) .partialProgressMaxCommits(10)) .append(); env.execute("Table Maintenance Job"); ``` ``` -------------------------------- ### Flink Streaming Read from Latest Snapshot Source: https://github.com/apache/iceberg/blob/main/docs/docs/flink-queries.md Starts a streaming read from the latest table snapshot and polls for new snapshots every 60 seconds. This example uses the FLIP-27 source interface and does not support CDC read yet. ```java StreamExecutionEnvironment env = StreamExecutionEnvironment.createLocalEnvironment(); TableLoader tableLoader = TableLoader.fromHadoopTable("hdfs://nn:8020/warehouse/path"); IcebergSource source = IcebergSource.forRowData() .tableLoader(tableLoader) .assignerFactory(new SimpleSplitAssignerFactory()) .streaming(true) .streamingStartingStrategy(StreamingStartingStrategy.INCREMENTAL_FROM_LATEST_SNAPSHOT) .monitorInterval(Duration.ofSeconds(60)) .build(); DataStream stream = env.fromSource( source, WatermarkStrategy.noWatermarks(), "My Iceberg Source", TypeInformation.of(RowData.class)); // Print all records to stdout. stream.print(); // Submit and execute this streaming read job. env.execute("Test Iceberg Streaming Read"); ``` -------------------------------- ### Show Makefile Help Source: https://github.com/apache/iceberg/blob/main/site/README.md Displays a list of available Makefile recipes for managing the documentation site. ```bash make help ``` -------------------------------- ### Snapshot Object Example Source: https://github.com/apache/iceberg/blob/main/format/spec.md Shows the structure of a snapshot object within the table metadata, including its ID, timestamp, summary, manifest list location, and schema ID. ```json [ { "snapshot-id": 3051729675574597004, "timestamp-ms": 1515100955770, "summary": { "operation": "append" }, "manifest-list": "s3://b/wh/.../s1.avro" "schema-id": 0 } ] ``` -------------------------------- ### Start Iceberg Flink Docker Compose Stack Source: https://github.com/apache/iceberg/blob/main/docker/iceberg-flink-quickstart/README.md Start the Docker containers defined in the docker-compose.yml file. The --build flag ensures that the image is built before starting the containers. ```bash docker compose -f docker/iceberg-flink-quickstart/docker-compose.yml up -d --build ``` -------------------------------- ### Create Table Like Existing Table Source: https://github.com/apache/iceberg/blob/main/docs/docs/flink-ddl.md Create a new table 'sample_like' with the same schema, partitioning, and properties as an existing table 'sample'. ```sql CREATE TABLE `hive_catalog`.`default`.`sample` ( id BIGINT COMMENT 'unique id', data STRING ); CREATE TABLE `hive_catalog`.`default`.`sample_like` LIKE `hive_catalog`.`default`.`sample`; ``` -------------------------------- ### Avro Schema for Map with Key and Value IDs Source: https://github.com/apache/iceberg/blob/main/format/spec.md Example of an Avro schema for a map type, including 'key-id' and 'value-id'. This is used when keys are not strings and for schema evolution. ```json { "type": "map", "values": "int", "key-id": 10, "value-id": 11 } ``` -------------------------------- ### Build and Stage Convenience Binaries Source: https://github.com/apache/iceberg/blob/main/site/docs/how-to-release.md Execute the stage-binaries.sh script to build and publish convenience binaries to a release staging repository. ```bash dev/stage-binaries.sh ``` -------------------------------- ### Install Apache Flink Dependency Source: https://github.com/apache/iceberg/blob/main/docs/docs/flink.md Use pip to install the required Apache Flink version. ```bash pip install apache-flink=={{ flinkVersion }} ``` -------------------------------- ### Example Stats Struct for an Integer Field Source: https://github.com/apache/iceberg/blob/main/format/spec.md Illustrates the structure of a statistics struct for a required integer field, showing common metrics like lower bound, upper bound, tight bounds, and value counts. Note that null_value_count, nan_value_count, and avg_value_size_in_bytes are omitted as they are not applicable to this field type. ```text 10_400: optional struct id (default null) { 10_401: optional int lower_bound; // type matches the field type (int) 10_402: optional int upper_bound; // type matches the field type (int) 10_403: optional boolean tight_bounds; 10_404: optional long value_count; // null_value_count is only used for optional fields // nan_value_count is only used for float and double // avg_value_size_in_bytes is only used for variable length types } ``` -------------------------------- ### SQL Representation Example Source: https://github.com/apache/iceberg/blob/main/site/docs/view-spec.md Example of a SQL representation for a view definition, including dialect and the SQL statement. ```json { "type": "sql", "sql": "SELECT\n COUNT(1), CAST(event_ts AS DATE)\nFROM events\nGROUP BY 2", "dialect": "spark" } ``` -------------------------------- ### Avro Schema for List Element with Element ID Source: https://github.com/apache/iceberg/blob/main/format/spec.md Example of an Avro schema for an array type, specifying an 'element-id' for the array's items. This is relevant for list schema evolution. ```json { "type": "array", "items": "int", "element-id": 9 } ``` -------------------------------- ### Initialize Nessie Catalog in Java Source: https://github.com/apache/iceberg/blob/main/docs/docs/nessie.md Load the Nessie catalog programmatically using CatalogUtil with required warehouse, reference, and URI settings. ```java Map options = new HashMap<>(); options.put("warehouse", "/path/to/warehouse"); options.put("ref", "main"); options.put("uri", "https://localhost:19120/api/v2"); Catalog nessieCatalog = CatalogUtil.loadCatalog("org.apache.iceberg.nessie.NessieCatalog", "nessie", options, hadoopConfig); ``` -------------------------------- ### Avro Schema for Struct Field with Field ID Source: https://github.com/apache/iceberg/blob/main/format/spec.md Example of an Avro record schema defining a struct field with a 'field-id'. This is used for schema evolution and identification. ```json { "type": "record", "name": "r1", "fields": [ { "name": "l", "type": ["null", "long"], "default": null, "field-id": 8 } ] } ``` -------------------------------- ### Serve Site in Development Mode Source: https://github.com/apache/iceberg/blob/main/site/README.md Builds and serves only the 'nightly' and 'latest' versions for faster iterative development. Skips historical versions. ```bash make serve-dev ``` -------------------------------- ### Primitive Type Serialization Source: https://github.com/apache/iceberg/blob/main/site/docs/spec.md Primitive data types are represented as JSON strings. For example, 'int' is serialized as "int", and 'boolean' as "boolean". ```json "unknown" "boolean" "int" "long" "float" "double" "date" "time" "timestamp" "timestamptz" "timestamp_ns" "timestamptz_ns" "string" "uuid" "binary" "variant" "geometry(srid:4326)" "geography(srid:4326, spherical)" ``` -------------------------------- ### LoggingMetricsReporter Commit Report Example Source: https://github.com/apache/iceberg/blob/main/docs/docs/metrics-reporting.md Example output of a CommitReport logged by the LoggingMetricsReporter. This report summarizes metrics from a commit operation. ```log INFO org.apache.iceberg.metrics.LoggingMetricsReporter - Received metrics report: CommitReport{ tableName=scan-planning-with-eq-and-pos-delete-files, snapshotId=1, sequenceNumber=1, operation=append, commitMetrics=CommitMetricsResult{ totalDuration=TimerResult{timeUnit=NANOSECONDS, totalDuration=PT0.098429626S, count=1}, attempts=CounterResult{unit=COUNT, value=1}, addedDataFiles=CounterResult{unit=COUNT, value=1}, removedDataFiles=null, totalDataFiles=CounterResult{unit=COUNT, value=1}, addedDeleteFiles=null, addedEqualityDeleteFiles=null, addedPositionalDeleteFiles=null, removedDeleteFiles=null, removedEqualityDeleteFiles=null, removedPositionalDeleteFiles=null, totalDeleteFiles=CounterResult{unit=COUNT, value=0}, addedRecords=CounterResult{unit=COUNT, value=1}, removedRecords=null, totalRecords=CounterResult{unit=COUNT, value=1}, addedFilesSizeInBytes=CounterResult{unit=BYTES, value=10}, removedFilesSizeInBytes=null, totalFilesSizeInBytes=CounterResult{unit=BYTES, value=10}, addedPositionalDeletes=null, removedPositionalDeletes=null, totalPositionalDeletes=CounterResult{unit=COUNT, value=0}, addedEqualityDeletes=null, removedEqualityDeletes=null, totalEqualityDeletes=CounterResult{unit=COUNT, value=0}}, metadata={ iceberg-version=Apache Iceberg 1.4.0-SNAPSHOT (commit 4868d2823004c8c256a50ea7c25cff94314cc135)}} ``` -------------------------------- ### Prefer Specific Method Names Over Getters Source: https://github.com/apache/iceberg/blob/main/site/docs/contribute.md Avoid using 'get' in method names unless the object is a Java bean. Replace 'get' with a more specific verb like 'find' or 'fetch'. If no specific verb applies or it's a simple getter, omit 'get' to shorten names. ```java // prefer exposing suppressFailure in method names public void sendMessageIgnoreFailure() { sendMessageInternal(true); } public void sendMessage() { sendMessageInternal(false); } private void sendMessageInternal(boolean suppressFailure) { ... } ``` -------------------------------- ### Serve Iceberg Site Locally Source: https://github.com/apache/iceberg/blob/main/site/README.md Builds the site and runs a local development server at http://localhost:8000. Ideal for previewing changes. ```bash make serve ``` -------------------------------- ### Build Source Release Artifacts Source: https://github.com/apache/iceberg/blob/main/site/docs/how-to-release.md Run the source-release.sh script with the release version and candidate number to create source release artifacts. Ensure you replace '' with your actual key ID. ```bash dev/source-release.sh -v 1.8.0 -r 0 -k ``` -------------------------------- ### Flink Streaming Read from Start Tag Source: https://github.com/apache/iceberg/blob/main/docs/docs/flink-queries.md Initiates a streaming read from an Iceberg table starting from a specified tag using the FlinkSource API. ```java StreamExecutionEnvironment env = StreamExecutionEnvironment.createLocalEnvironment(); TableLoader tableLoader = TableLoader.fromHadoopTable("hdfs://nn:8020/warehouse/path"); DataStream batch = FlinkSource.forRowData() .env(env) .tableLoader(tableLoader) .streaming(true) .startTag("test-tag") .build(); ``` -------------------------------- ### LoggingMetricsReporter Scan Report Example Source: https://github.com/apache/iceberg/blob/main/docs/docs/metrics-reporting.md Example output of a ScanReport logged by the LoggingMetricsReporter. This report details metrics related to a table scan operation. ```log INFO org.apache.iceberg.metrics.LoggingMetricsReporter - Received metrics report: ScanReport{ tableName=scan-planning-with-eq-and-pos-delete-files, snapshotId=2, filter=ref(name="data") == "(hash-27fa7cc0)", schemaId=0, projectedFieldIds=[1, 2], projectedFieldNames=[id, data], scanMetrics=ScanMetricsResult{ totalPlanningDuration=TimerResult{timeUnit=NANOSECONDS, totalDuration=PT0.026569404S, count=1}, resultDataFiles=CounterResult{unit=COUNT, value=1}, resultDeleteFiles=CounterResult{unit=COUNT, value=2}, totalDataManifests=CounterResult{unit=COUNT, value=1}, totalDeleteManifests=CounterResult{unit=COUNT, value=1}, scannedDataManifests=CounterResult{unit=COUNT, value=1}, skippedDataManifests=CounterResult{unit=COUNT, value=0}, totalFileSizeInBytes=CounterResult{unit=BYTES, value=10}, totalDeleteFileSizeInBytes=CounterResult{unit=BYTES, value=20}, skippedDataFiles=CounterResult{unit=COUNT, value=0}, skippedDeleteFiles=CounterResult{unit=COUNT, value=0}, scannedDeleteManifests=CounterResult{unit=COUNT, value=1}, skippedDeleteManifests=CounterResult{unit=COUNT, value=0}, indexedDeleteFiles=CounterResult{unit=COUNT, value=2}, equalityDeleteFiles=CounterResult{unit=COUNT, value=1}, positionalDeleteFiles=CounterResult{unit=COUNT, value=1}}, metadata={ iceberg-version=Apache Iceberg 1.4.0-SNAPSHOT (commit 4868d2823004c8c256a50ea7c25cff94314cc135)}} ```