### Install Sphinx and Theme on Ubuntu Source: https://github.com/locationtech/geotrellis/blob/master/docs/CONTRIBUTING.md Installs the necessary Sphinx documentation generator and Read the Docs theme for Ubuntu 16.04. ```bash sudo apt-get install python-sphinx python-sphinx-rtd-theme ``` -------------------------------- ### Install Sphinx and Theme on Arch Linux Source: https://github.com/locationtech/geotrellis/blob/master/docs/CONTRIBUTING.md Installs the necessary Sphinx documentation generator and Read the Docs theme for Arch Linux. ```bash sudo pacman -S python-sphinx python-sphinx_rtd_theme ``` -------------------------------- ### Install Sphinx and Theme using Pip Source: https://github.com/locationtech/geotrellis/blob/master/docs/CONTRIBUTING.md Installs the Sphinx documentation generator and Read the Docs theme using pip, suitable for macOS or other environments where pip is preferred. ```bash pip install sphinx sphinx_rtd_theme ``` -------------------------------- ### ETL Pipeline Example Source: https://github.com/locationtech/geotrellis/blob/master/docs/architecture/005-etl-pipeline.md An example of a complete ETL pipeline configuration, demonstrating a sequence of read, group, reproject, tile, and write steps. ```json [ { "type": "read.hadoop", "profile": "credentials profile name", "path": "{uri-path}", "crs": "input source CRS", "tag": "A" }, { "type": "read.s3", "profile": "credentials profile name", "path": "{uri-path}", "crs": "input source CRS", "tag": "B" }, { "type": "read.s3", "profile": "credentials profile name", "path": "{uri-path}", "crs": "input source CRS", "tag": "С" }, { "type": "transform.group", "tags": ["A", "B"], // no more tags A and B in the context of this pipeline "tag": "D" }, { "type": "transform.reproject.buffered", "crs": "{EPSG code | EPSG name | proj4 string}" }, { "type": "transform.tile", "resampleMethod": "{nearest-neighbor | bilinear | cubic-convolution | cubic-spline | lanczos}", "layoutScheme": "zoomed", "tileSize": 256, "cellSize": { "width": 0.5, "height": 0.5 }, "partitions": 5000 }, { "type": "write.render", "format": "{tiff | png}", "path": "{path | pattern}" }, { "type": "write.{save | update | reindex}", ``` -------------------------------- ### Copy Makefile and global.sbt Templates Source: https://github.com/locationtech/geotrellis/blob/master/publish/README.md Copy the template files for Makefile and global.sbt to your project directory to begin the publishing setup. ```bash cp Makefile.template Makefile cp global.sbt.template global.sbt ``` -------------------------------- ### Tile Files Listing Example Source: https://github.com/locationtech/geotrellis/blob/master/docs/guide/core-concepts.md Demonstrates how to list serialized tile files within a specific zoom level directory of a catalog and check their sizes. ```console $ ls attributes/ landsat/ $ cd landsat/6/ $ ls 1984 1985 1986 1987 $ du -sh * 12K 1984 8.0K 1985 44K 1986 16K 1987 ``` -------------------------------- ### Start Docusaurus Dev Server Source: https://github.com/locationtech/geotrellis/blob/master/website/README.md Start the Docusaurus development server to view the documentation site locally. This command will automatically recompile docs when changes are detected. ```shell yarn start ``` -------------------------------- ### Install Project Dependencies Source: https://github.com/locationtech/geotrellis/blob/master/website/README.md Install project dependencies using yarn. Ensure you have Node 12+ installed. ```shell yarn install --pure-lockfile ``` -------------------------------- ### Executing a Pipeline from JSON Source: https://github.com/locationtech/geotrellis/blob/master/docs/architecture/005-etl-pipeline.md Example of how to execute a defined pipeline using a JSON configuration. This demonstrates a high-level API for pipeline execution. ```scala // json can be AST / or a JSON string val rdd: TileLayerRDD[SpatialKey] = Pipeline.execute[TileLayerRDD[SpatialKey]](json) ``` -------------------------------- ### Create an IntArrayTile Source: https://github.com/locationtech/geotrellis/blob/master/docs/tutorials/quickstart.md Create an example Tile using an integer array, specifying dimensions. Note that NODATA values are handled based on the CellType. ```scala scala> val nd = NODATA nd: Int = -2147483648 ``` ```scala scala> val input = Array[Int]( nd, 7, 1, 1, 3, 5, 9, 8, 2, 9, 1, 1, 2, 2, 2, 4, 3, 5, 3, 8, 1, 3, 3, 3, 1, 2, 2, 2, 4, 7, 1, nd, 1, 8, 4, 3) input: Array[Int] = Array(-2147483648, 7, 1, 1, 3, 5, 9, 8, 2, 9, 1, 1, 2, 2, 2, 4, 3, 5, 3, 8, 1, 3, 3, 3, 1, 2, 2, 2, 4, 7, 1, -2147483648, 1, 8, 4, 3) ``` ```scala scala> val iat = IntArrayTile(input, 9, 4) // 9 and 4 here specify columns and rows iat: geotrellis.raster.IntArrayTile = IntArrayTile(I@278434d0,9,4) ``` -------------------------------- ### Running Unit Tests for Custom Formats Source: https://github.com/locationtech/geotrellis/blob/master/docs/guide/extending-geotrellis.md Execute unit tests for your custom `JsonFormat` and `KeyIndexRegistrator` implementations using sbt. This example shows how to target a specific test class. ```bash geotrellis > project doc-examples doc-examples > testOnly geotrellis.doc.examples.spark.ShardingKeyIndexSpec ``` -------------------------------- ### Small Files Benchmark Output Example Source: https://github.com/locationtech/geotrellis/blob/master/docs/raster/WindowedGeoTiffReaderDemo.md This output displays the results of the small files benchmark, comparing the performance of Windowed GeoTiff Reader (WGR) against traditional cropping methods for Byte and Float64 bandtypes at various percentages. ```text Running benchmarks for Reading in small Extents of small files... Name ms linear runtime 5%, WGR, Byte 1.53 == 5%, cropping, Byte 1.97 == 5%, WGR, Float64 2.57 === 5%, cropping, Float64 13.68 ================= 10%, WGR, Byte 1.56 == 10%, cropping, Byte 1.89 == 10%, WGR, Float64 2.93 === 10%, cropping, Float64 16.23 ===================== 15%, WGR, Byte 1.31 = 15%, cropping, Byte 2.17 == 15%, WGR, Float64 3.34 ==== 15%, cropping, Float64 13.76 ================== Running benchmarks for Reading in large Extents of small files... Name ms linear runtime 90%, WGR, Byte 1.90 == 90%, cropping, Byte 2.11 == 90%, WGR, Float64 11.22 ============= 90%, cropping, Float64 15.64 =================== 95%, WGR, Byte 2.05 == 95%, cropping, Byte 2.11 == 95%, WGR, Float64 11.82 ============== 95%, cropping, Float64 15.70 =================== 100%, WGR, Byte 1.69 == 100%, cropping, Byte 1.95 == 100%, WGR, Float64 12.98 ================ 100%, cropping, Float64 15.33 ================== ``` -------------------------------- ### Create and Render an ASCII Raster Tile Source: https://github.com/locationtech/geotrellis/blob/master/README.md This example demonstrates creating an IntArrayTile from an array of integers, including handling NoData values. It then renders the tile to an ASCII string representation using a specified palette. ```scala import geotrellis.raster._ import geotrellis.raster.render.ascii._ val nd = NODATA val input = Array[Int]( nd, 7, 1, 1, 3, 5, 9, 8, 2, 9, 1, 1, 2, 2, 2, 4, 3, 5, 3, 8, 1, 3, 3, 3, 1, 2, 2, 2, 4, 7, 1, nd, 1, 8, 4, 3) val iat = IntArrayTile(input, 9, 4) // 9 and 4 here specify columns and rows iat.renderAscii(AsciiArtEncoder.Palette.STIPLED) ``` -------------------------------- ### S3 Layer Metadata Header Example Source: https://github.com/locationtech/geotrellis/blob/master/docs/guide/core-concepts.md Shows the 'header' block within a metadata JSON for a layer ingested to S3, specifying the S3 bucket and key for tile storage. ```javascript ... "header": { "format": "s3", "key": "catalog/nlcd-tms-epsg3857/6", "keyClass": "geotrellis.spark.SpatialKey", "valueClass": "geotrellis.raster.Tile", "bucket": "azavea-datahub" }, ... ``` -------------------------------- ### Catalog Directory Structure Example Source: https://github.com/locationtech/geotrellis/blob/master/docs/guide/core-concepts.md Illustrates the typical directory layout of a Geotrellis catalog on the file system, showing attribute metadata and layer data organized by zoom level. ```text . ├── attributes │ ├── landsat.__0__.__metadata.json │ ├── landsat.__10__.__metadata.json │ ├── landsat.__11__.__metadata.json │ ├── landsat.__12__.__metadata.json │ ├── landsat.__13__.__metadata.json │ ├── landsat.__1__.__metadata.json │ ├── landsat.__2__.__metadata.json │ ├── landsat.__3__.__metadata.json │ ├── landsat.__4__.__metadata.json │ ├── landsat.__5__.__metadata.json │ ├── landsat.__6__.__metadata.json │ ├── landsat.__7__.__metadata.json │ ├── landsat.__8__.__metadata.json │ └── landsat.__9__.__metadata.json └── landsat ├── 0 ├── 1 ├── 10 ├── 11 ├── 12 ├── 13 ├── 2 ├── 3 ├── 4 ├── 5 ├── 6 ├── 7 ├── 8 └── 9 16 directories, 14 files ``` -------------------------------- ### ARG Raster Metadata Example Source: https://github.com/locationtech/geotrellis/wiki/ARG-Specification-[DEPRECATED] This JSON structure defines the metadata for an ARG raster. Ensure all required keys are present for proper interpretation. ```javascript { "layer": "philly rainfall", "type": "arg", "datatype": "int8", "xmin": -8507736.525864778, "ymin": 4847928.144104313, "xmax": -8457736.525864778, "ymax": 4897928.144104313, "cellwidth": 10.0, "cellheight": 10.0, "rows": 5000, "cols": 5000 } ``` -------------------------------- ### For-Loop Translation Example Source: https://github.com/locationtech/geotrellis/blob/master/docs/architecture/high-performance-scala.md Shows the potential translation of a simple nested `for`-loop into nested `foreach` calls, highlighting the overhead that `cfor` aims to avoid. ```scala (0 to 100).foreach({ x => (0 to 100).foreach({ y => println(x+y) }) }) ``` -------------------------------- ### NDVI Calculation with Spark and GeoTrellis Source: https://context7.com/locationtech/geotrellis/llms.txt An end-to-end example demonstrating spatial and temporal layer querying, NDVI band math, temporal max reduction, and GeoTiff output using GeoTrellis and Spark. Requires a FilteringLayerReader and specific layer IDs. ```scala import geotrellis.raster._ import geotrellis.raster.io.geotiff._ import geotrellis.spark._ import geotrellis.spark.io._ import geotrellis.vector._ import org.joda.time.DateTime val region: MultiPolygon = ??? // study area val layerReader: FilteringLayerReader[LayerId] = ??? // your backend reader val layerId = LayerId("Landsat8-Bands", zoom = 18) // Query multiband tiles intersecting region within a date range val queryResult: MultibandTileLayerRDD[SpaceTimeKey] = layerReader .query[SpaceTimeKey, MultibandTile, TileLayerMetadata[SpaceTimeKey]](layerId) .where(Intersects(region)) .where(Between(new DateTime(2016, 3, 1, 0, 0), new DateTime(2016, 4, 1, 0, 0))) .result // Mask to the exact polygon boundary, compute max NDVI across dates val maxNdvi: Raster[Tile] = queryResult .mask(region) .withContext { rdd => rdd .mapValues { tile => // Band 4 = Red, Band 5 = NIR (Landsat 8 Surface Reflectance) tile.convert(DoubleConstantNoDataCellType).combine(4, 5) { (red, nir) => if (isNoData(red) || isNoData(nir)) NODATA.toDouble else (nir - red) / (nir + red) } } .map { case (key, tile) => (key.getComponent[SpatialKey], tile) } .reduceByKey(_.localMax(_)) } .stitch // Write result as Cloud-Optimized GeoTiff GeoTiff(maxNdvi, queryResult.metadata.crs).write("/output/max_ndvi.tif") ``` -------------------------------- ### Generate Image Pyramid from Input Dataset Source: https://github.com/locationtech/geotrellis/wiki/GeoTrellis-3.0-Examples Generates a multi-resolution image pyramid from an RDD of raster tiles. This example shows how to reproject, define a layout scheme, and save each zoom level as a PNG to S3. ```scala import geotrellis.layer._ import geotrellis.proj4.WebMercator import geotrellis.raster._ import geotrellis.raster.reproject.Reproject import geotrellis.spark._ import geotrellis.spark.pyramid._ import geotrellis.spark.store.s3._ import org.apache.spark.rdd.RDD // There are a number of ways to generate this input // which is out of scope of this example val tileRdd: RDD[(SpatialKey, Tile)] = ??? // Use whichever of the overloads correspond to the // information you know about your tileRdd val metadata: TileLayerMetadata[SpatialKey] = tileRdd.collectMetadata(???) // Placeholder for actual metadata collection val resampleOptions = Reproject.Options.DEFAULT // This is the layout scheme compatible with tiled slippy maps val pyramidLayoutScheme = ZoomedLayoutScheme(WebMercator, 256) // Convert your RDD to the correct layout/projection val (targetZoom, tileLayerRdd) = TileLayerRDD(tileRdd, metadata) .reproject(pyramidLayoutScheme, resampleOptions) // You can explicitly set the zoom levels to generate, // or let GeoTrellis figure out the optimal zoom levels, // as is done here. val pyramid = Pyramid.fromLayerRDD(tileLayerRdd, Some(targetZoom), None) pyramid.levels.foreach { // Write each zoom level sequentially case (zoom, tileRdd) => { // Feel free to generate a fancier colormap here val colorRamp = ColorRamps.BlueToOrange val histogram = tileRdd.histogram(colorRamp.numStops) val colorMap = colorRamp.toColorMap(histogram) val keyToPath = (k: SpatialKey) => s"s3://bucket/key/path/${zoom}/${k.col}/${k.row}.png" tileRdd.map { case (key, tile) => { (key, tile.renderPng(colorMap).bytes) } }.saveToS3(keyToPath) } } ``` -------------------------------- ### SBT Lighter Plugin Configuration for EMR GDAL Deployment Source: https://github.com/locationtech/geotrellis/blob/master/docs-mdoc/emr-deployments.md Configure the SBT Lighter plugin to specify EMR release, region, applications, and bootstrap actions for installing GDAL. This setup ensures GeoTrellis can utilize GDAL on EMR. ```scala /** addSbtPlugin("net.pishen" % "sbt-lighter" % "1.2.0") */ import sbtlighter._ LighterPlugin.disable lazy val EMRSettings = LighterPlugin.baseSettings ++ Seq( sparkEmrRelease := "emr-6.0.0", sparkAwsRegion := "us-east-1", sparkEmrApplications := Seq("Hadoop", "Spark", "Ganglia", "Zeppelin"), sparkEmrBootstrap := List( BootstrapAction( "Install GDAL dependencies", "s3://geotrellis-test/emr-gdal/bootstrap.sh", "3.1.2" ) ), sparkS3JarFolder := "s3://geotrellis-test/rastersource-performance/jars", sparkInstanceCount := 11, sparkMasterType := "i3.xlarge", sparkCoreType := "i3.xlarge", sparkMasterPrice := Some(0.5), sparkCorePrice := Some(0.5), sparkClusterName := s"GeoTrellis VLM Performance ${sys.env.getOrElse("USER", "")}", sparkEmrServiceRole := "EMR_DefaultRole", sparkInstanceRole := "EMR_EC2_DefaultRole", sparkMasterEbsSize := None, // Some(64) sparkCoreEbsSize := None, // Some(64) sparkJobFlowInstancesConfig := sparkJobFlowInstancesConfig.value.withEc2KeyName("geotrellis-emr"), sparkS3LogUri := Some("s3://geotrellis-test/rastersource-performance/logs"), sparkEmrConfigs := List( EmrConfig("spark").withProperties( "maximizeResourceAllocation" -> "false" // be careful with setting this param to true ), EmrConfig("spark-defaults").withProperties( "spark.driver.maxResultSize" -> "4200M", "spark.dynamicAllocation.enabled" -> "true", "spark.shuffle.service.enabled" -> "true", "spark.shuffle.compress" -> "true", "spark.shuffle.spill.compress" -> "true", "spark.rdd.compress" -> "true", "spark.driver.extraJavaOptions" ->"-XX:+UseParallelGC -XX:+UseParallelOldGC -XX:OnOutOfMemoryError='kill -9 %p'", "spark.executor.extraJavaOptions" -> "-XX:+UseParallelGC -XX:+UseParallelOldGC -XX:OnOutOfMemoryError='kill -9 %p'", "spark.yarn.appMasterEnv.LD_LIBRARY_PATH" -> "/usr/local/miniconda/lib/:/usr/local/lib", "spark.executorEnv.LD_LIBRARY_PATH" -> "/usr/local/miniconda/lib/:/usr/local/lib" ), EmrConfig("yarn-site").withProperties( "yarn.resourcemanager.am.max-attempts" -> "1", "yarn.nodemanager.vmem-check-enabled" -> "false", "yarn.nodemanager.pmem-check-enabled" -> "false" ) ) ) ``` -------------------------------- ### Initialize File System Layer IO Components Source: https://github.com/locationtech/geotrellis/blob/master/docs/guide/core-concepts.md Set up the AttributeStore, LayerReader, and LayerWriter for the file system backend. Requires a catalog path. ```scala import geotrellis.spark._ import geotrellis.spark.io._ import geotrellis.spark.io.file._ val catalogPath: String = ... /* Some location on your computer */ val store: AttributeStore = FileAttributeStore(catalogPath) val reader = FileLayerReader(store) val writer = FileLayerWriter(store) ``` -------------------------------- ### Calculate Focal Mean on a Raster Tile Source: https://github.com/locationtech/geotrellis/blob/master/README.md This example shows how to apply a focal mean operation to a raster tile using a specified neighborhood. The result is a new tile with the calculated mean values. The example also verifies a specific pixel's calculated value. ```scala import geotrellis.raster.mapalgebra.focal._ val focalNeighborhood = Square(1) // a 3x3 square neighborhood val meanTile = iat.focalMean(focalNeighborhood) meanTile.getDouble(0, 0) // Should equal (1 + 7 + 9) / 3 ``` -------------------------------- ### Run All Benchmarks (sbt) Source: https://github.com/locationtech/geotrellis/blob/master/bench/README.md Execute all JMH benchmarks for the bench project from the project root directory. ```bash sbt bench/bench ``` -------------------------------- ### Check Java Version Source: https://github.com/locationtech/geotrellis/blob/master/docs/tutorials/setup.md Verify that Java 8 or a compatible version is installed. GeoTrellis requires Java 8 or higher. ```console javac -version ``` -------------------------------- ### Initialize File System Backend for Local Operations Source: https://github.com/locationtech/geotrellis/blob/master/docs/guide/tile-backends.md The File System backend is suitable for local testing, data ingestion, or processing. It requires a catalog path and provides simple initialization for AttributeStore, LayerReader, and FileLayerWriter. ```scala import geotrellis.spark._ import geotrellis.spark.io._ import geotrellis.spark.io.file._ val catalogPath: String = ... val store: AttributeStore = FileAttributeStore(catalogPath) val reader = FileLayerReader(store) val writer = FileLayerWriter(store) ``` -------------------------------- ### Create Extent and Raster Instances Source: https://github.com/locationtech/geotrellis/blob/master/docs/guide/core-concepts.md Shows how to define an Extent and then construct a Raster using a tile and the defined Extent. ```scala scala> Extent(0, 0, 1, 1) res4: geotrellis.vector.Extent = Extent(0.0,0.0,1.0,1.0) ``` ```scala scala> Raster(res2, res4) res5: geotrellis.raster.Raster = Raster(IntArrayTile([I@7b47ab7,1,3),Extent(0.0,0.0,1.0,1.0)) ``` -------------------------------- ### Modify Metadata After RDD Operation Source: https://github.com/locationtech/geotrellis/blob/master/docs/guide/spark.md Use `.mapContext` to change metadata after an RDD operation, for example, setting the cell type. ```scala val rddWithContext3: RDD[(SpatialKey, Tile)] with Metadata[TileLayerMetadata] = rdd1 .withContext { _ localEqual 123 } .mapContext { metadata: TileLayerMetadata => metadata.copy(cellType = TypeBit) } ``` -------------------------------- ### Build GeoTrellis Documentation Source: https://github.com/locationtech/geotrellis/blob/master/docs/CONTRIBUTING.md Navigates to the docs directory and builds the HTML documentation using make. ```bash make html ``` -------------------------------- ### Typeclass Constraint Example Source: https://github.com/locationtech/geotrellis/blob/master/docs/guide/core-concepts.md Illustrates how typeclass constraints are declared in Scala method signatures using the ':' syntax, which is syntactic sugar for implicit parameters. ```scala class Foo[A: Order](a: A) { ... } ``` -------------------------------- ### Run All Benchmarks (sbt session) Source: https://github.com/locationtech/geotrellis/blob/master/bench/README.md Execute all JMH benchmarks for the bench project within an sbt interactive session. ```bash geotrellis> project bench bench> bench ``` -------------------------------- ### JSON Type Representation Source: https://github.com/locationtech/geotrellis/blob/master/docs/architecture/005-etl-pipeline.md Example of how a type is represented in JSON for pipeline steps. The 'type' field includes package information for class identification. ```json "type": "geotrellis.pipeline.singleband.temporal.Reproject" ``` -------------------------------- ### Define Display Extension Methods Trait Source: https://github.com/locationtech/geotrellis/blob/master/docs/tutorials/util-intro.md Define a trait that extends MethodExtensions to declare new methods. This example shows how to add a 'display' method. ```scala package geotrellis.raster.display import geotrellis.util.MethodExtensions trait DisplayMethods[T <: AnyVal] MethodExtensions[T] { def display(): Unit = println(self) } ``` -------------------------------- ### Read Step Configuration Source: https://github.com/locationtech/geotrellis/blob/master/docs/architecture/005-etl-pipeline.md Defines how to read data from various sources like Hadoop, S3, or local files. Specify credentials, path, CRS, and an optional tag for the layer. ```json { "type": "read.{hadoop | s3 | file}", "profile": "credentials profile name", "path": "{uri-path}", "crs": "input source CRS", "tag": "tag-name" // tag means current layer name } ``` -------------------------------- ### Run Small Files Benchmark Source: https://github.com/locationtech/geotrellis/blob/master/docs/raster/WindowedGeoTiffReaderDemo.md This command initiates the benchmark tests specifically for the WindowedGeoTiffReader on small GeoTiff files. It targets tests related to 'SmallFilesBenchmark'. ```bash > ./sbt "project windowedreader-benchmark" "test-only *SmallFilesBenchmark*" ``` -------------------------------- ### ShardingKeyIndex with Inner KeyIndex Source: https://github.com/locationtech/geotrellis/blob/master/docs/guide/extending-geotrellis.md An example of a generic ShardingKeyIndex that wraps an inner KeyIndex, allowing for sharding logic to be applied on top of an existing indexing strategy. ```scala class ShardingKeyIndex[K](val inner: KeyIndex[K], val shardCount: Int) extends KeyIndex[K] { ... } ``` -------------------------------- ### Publish to Sonatype Staging Repository Source: https://github.com/locationtech/geotrellis/blob/master/publish/README.md Execute 'make publish' to run the 'publish-to-sonatype.sh' script within the container. This pushes the release artifacts to the Sonatype staging repository. ```bash make publish ``` -------------------------------- ### Converting to and from JSON with Spray JsonFormat Source: https://github.com/locationtech/geotrellis/blob/master/docs/guide/core-concepts.md Example of using the JsonFormat typeclass from the Spray library to convert a Scala object to a JSON Value and then back to the original type. ```scala def toJsonAndBack[A: JsonFormat](a: A): A = { val json: Value = a.toJson json.convertTo[A] } ``` -------------------------------- ### Island Draw Down Attribute Function Source: https://github.com/locationtech/geotrellis/blob/master/docs/guide/rasters.md Calculates the estimated groundwater level contribution of an island based on its coordinates. This function is used as an example of a custom attribute function. ```scala /** Estimate of the island's contribution to groundwater level * //exp{-1.0 * (sqrt((s1-560)^2 + (s2-390)^2) / 35)^8 } */ val islandDrawDown: Double = math.exp(-1 * math.pow(math.sqrt(math.pow(x - 560, 2) + math.pow(y - 390, 2)) / 35, 8)) ``` -------------------------------- ### Initialize SparkContext for Kernel Density Source: https://github.com/locationtech/geotrellis/blob/master/docs/tutorials/kernel-density.md Set up a SparkConf and SparkContext to enable distributed processing. This is the initial step for any Spark application. ```scala import org.apache.spark.{SparkConf, SparkContext} val conf = new SparkConf().setMaster("local").setAppName("Kernel Density") val sc = new SparkContext(conf) ``` -------------------------------- ### Create IntArrayTile Instances Source: https://github.com/locationtech/geotrellis/blob/master/docs/guide/core-concepts.md Demonstrates creating IntArrayTile instances with different dimensions. Ensure the array size matches the product of columns and rows. ```scala import geotrellis.raster._ import geotrellis.vector._ scala> IntArrayTile(Array(1,2,3),1,3) res0: geotrellis.raster.IntArrayTile = IntArrayTile([S@338514ad,1,3) ``` ```scala scala> IntArrayTile(Array(1,2,3),3,1) res1: geotrellis.raster.IntArrayTile = IntArrayTile([S@736a81de,3,1) ``` ```scala scala> IntArrayTile(Array(1,2,3,4,5,6,7,8,9),3,3) res2: geotrellis.raster.IntArrayTile = IntArrayTile([I@5466441b,3,3) ``` -------------------------------- ### Define Component Trait in Scala Source: https://github.com/locationtech/geotrellis/blob/master/docs/guide/core-concepts.md Component is a basic Lens, providing generic get and set functions for data structures, especially useful for nested data. ```scala trait Component[T, C] extends GetComponent[T, C] with SetComponent[T, C] ``` -------------------------------- ### Polygonal Sum Summary Operation Source: https://github.com/locationtech/geotrellis/blob/master/docs/guide/rasters.md Performs a sum aggregation on raster pixels that fall within a specified polygon. This example demonstrates using the SumVisitor for raster data. ```scala import geotrellis.raster._ // This import extends Raster with the polygonalSummary method import geotrellis.raster.summary.polygonal._ import geotrellis.raster.summary.polygonal.visitors.SumVisitor import geotrellis.raster.summary.types.SumValue // Source these from your application val raster: Raster[Tile] = ??? val polygon: Polygon = ??? val summary = raster.polygonalSummary(polygon, SumVisitor) match summary { case Summary(result: SumValue) => println("Sum across polygon: ${result}") case NoIntersection => println("No Intersection!") } ``` -------------------------------- ### Configure Sonatype Credentials Source: https://github.com/locationtech/geotrellis/blob/master/publish/README.md Provide your Sonatype credentials in the './sonatype.sbt' file. Ensure the 'user' and 'password' fields are updated. ```scala realm=Sonatype Nexus Repository Manager host=oss.sonatype.org user=username password=password ``` -------------------------------- ### Check Project Dependencies Against Licenses Source: https://github.com/locationtech/geotrellis/blob/master/docs/CONTRIBUTING.md Checks the generated dependencies list against license requirements using the DashLicenses JAR. ```bash cat target/dependencies-list.txt | java -jar ./org.eclipse.dash.licenses-1.0.3-20230602.055020-10.jar - ``` -------------------------------- ### Calculate Maximum Temperature Per Pixel Source: https://github.com/locationtech/geotrellis/blob/master/docs/guide/spark.md Example of reducing a time series of temperature tiles to a single layer of maximum temperatures per pixel using `reduceByKey`. ```scala import geotrellis.raster._ import geotrellis.spark._ import geotrellis.util._ import org.apache.spark.rdd.RDD val temperaturePerMonth: TileLayerRDD[SpaceTimeKey] = ??? val maximumTemperature: RDD[(SpatialKey, Tile)] = temperaturePerMonth .map { case (key, tile) => // Get the spatial component of the SpaceTimeKey, which turns it into SpatialKey (key.getComponent[SpatialKey], tile) } // Now we have all the tiles that cover the same area with the same key. // Simply reduce by the key with a localMax .reduceByKey(_.localMax(_)) ``` -------------------------------- ### Sample Points and Locations for Kriging Source: https://github.com/locationtech/geotrellis/blob/master/docs/guide/vectors.md Initializes arrays of sample points with data and points for which kriging predictions are desired. ```scala //Array of sample points with given data val points: Array[PointFeature[Double]] = ... //Array of points to be kriged val location: Array[Point] = ... ``` -------------------------------- ### Import GeoTrellis Dependencies in Zeppelin Source: https://github.com/locationtech/geotrellis/blob/master/scripts/emr/README.md Example of importing GeoTrellis dependencies within a Zeppelin notebook. Ensure the GeoTrellis JAR has been added to the Zeppelin interpreter's classpath. ```scala import geotrellis.spark._ import geotrellis.raster._ ``` -------------------------------- ### Constructing a COG with Sidecar Overviews Source: https://github.com/locationtech/geotrellis/blob/master/docs/guide/core-concepts.md Demonstrates how to create a COG layer that references external overview files. This is useful when you cannot modify the original GeoTIFF to embed overviews. ```scala val mainTile = SinglebandGeoTiff(path = file:///path/to/my/file.tif) val overview1 = SinglebandGeoTiff(path = file:///path/to/my/file.tif.ovr1) val overview2 = SinglebandGeoTiff(path = file:///path/to/my/file.tif.ovr2) val tileWithOverview = mainTile.withOverviews(List(overview1, overview2)) ``` -------------------------------- ### Load Raster Data with RasterSource Source: https://github.com/locationtech/geotrellis/blob/master/docs-mdoc/getting-started.md Use the RasterSource API to easily load raster data, such as GeoTIFF files. This example demonstrates loading an aspect raster and accessing its extent. ```scala import geotrellis.raster._ val rs = RasterSource("gtiff+file://raster/data/aspect.tif") rs.extent ``` -------------------------------- ### Configure HBase Layer Reader and Writer Source: https://github.com/locationtech/geotrellis/blob/master/docs/guide/tile-backends.md Set up HBaseAttributeStore, HBaseLayerReader, and HBaseLayerWriter for reading and writing tile data to HBase. Requires an implicit SparkContext. ```scala import geotrellis.spark._ import geotrellis.spark.io._ import geotrellis.spark.io.hbase._ /* from the `geotrellis-hbase package */ val instance: HBaseInstance = ... val attrTable: String = ... val dataTable: String = ... implicit val sc: SparkContext = ... val store: AttributeStore = HBaseAttributeStore(instance, attrTable) val reader = HBaseLayerReader(store) /* Needs the implicit SparkContext */ val writer = HBaseLayerWriter(store, dataTable) ``` -------------------------------- ### Manage GeoTrellis Dependencies in build.sbt Source: https://github.com/locationtech/geotrellis/blob/master/docs/tutorials/setup.md Add or update GeoTrellis modules to your `build.sbt` file to include them in your project. This example shows how to add `geotrellis-spark`, `geotrellis-s3-spark`, and Spark core dependencies. ```scala libraryDependencies ++= Seq( "org.locationtech.geotrellis" %% "geotrellis-spark" % "3.0.0", "org.locationtech.geotrellis" %% "geotrellis-s3-spark" % "3.0.0", // now we can use the Amazon S3 store! "org.apache.spark" %% "spark-core" % "2.4.3" % "provided", "org.scalatest" %% "scalatest" % "3.0.8" % "test" ) ``` -------------------------------- ### Create AttributeStore, LayerReader, and LayerWriter from URI Source: https://github.com/locationtech/geotrellis/blob/master/docs/guide/tile-backends.md Instances of AttributeStore, LayerReader, and LayerWriter can be created directly from a URI. The scheme determines the backend, and query parameters can be used for configuration. ```scala import java.net.URI val uri = new URI("s3://bucket/catalog") val store = AttributeStore(uri) val reader = LayerReader(uri) val writer = LayerWriter(uri) val values = ValueReader(uri) ``` -------------------------------- ### Get Extent of a Tile using SpatialKey Source: https://github.com/locationtech/geotrellis/blob/master/docs/guide/examples.md Determine the geographic Extent covered by a given (SpatialKey, Tile) pair using the LayoutDefinition. Supports versions of GeoTrellis 1.2 and earlier. ```scala val layer: TileLayerRDD[SpatialKey] = ... /* Output of previous work */ val layout: LayoutDefinition = layer.metadata.layout val (key, tile): (SpatialKey, Tile) = ... val extent1: Extent = key.extent(layout) /* As of GeoTrellis 1.2 */ val extent2: Extent = layout.mapTransform(key) /* For versions < 1.2 */ ``` -------------------------------- ### File System Layer Metadata JSON Source: https://github.com/locationtech/geotrellis/blob/master/docs/guide/core-concepts.md Example of a metadata JSON file for a layer stored on the file system, detailing format, key/value classes, path, extent, and key index. ```json [ { "name": "landsat", "zoom": 6 }, { "header": { "format": "file", "keyClass": "geotrellis.spark.SpatialKey", "valueClass": "geotrellis.raster.MultibandTile", "path": "landsat/6" }, "metadata": { "extent": { "xmin": 15454940.911194608, "ymin": 4146935.160646211, "xmax": 15762790.223459147, "ymax": 4454355.929947533 }, "layoutDefinition": { ... } }, ... // more here "keyIndex": { "type": "zorder", "properties": { "keyBounds": { "minKey": { "col": 56, "row": 24 }, "maxKey": { "col": 57, "row": 25 } } } }, ... // more here } ] ``` -------------------------------- ### Request IP Team Review for Project Dependencies Source: https://github.com/locationtech/geotrellis/blob/master/docs/CONTRIBUTING.md Submits the dependencies list for an IP team review, including project name and GitLab authentication token. ```bash cat target/dependencies-list.txt | java -jar ./org.eclipse.dash.licenses-1.0.3-20230602.055020-10.jar - -review -project locationtech.geotrellis -token ```