### Quick Joern Installation Script Source: https://github.com/joernio/joern/blob/master/README.md Download, make executable, and run the Joern installation script for a quick setup. This is the primary method for installing Joern. ```bash wget https://github.com/joernio/joern/releases/latest/download/joern-install.sh chmod +x ./joern-install.sh sudo ./joern-install.sh joern ██╗ ██████╗ ███████╗██████╗ ███╗ ██╗ ██║██╔═══██╗██╔════╝██╔══██╗████╗ ██║ ██║██║ ██║█████╗ ██████╔╝██╔██╗ ██║ ██ ██║██║ ██║██╔══╝ ██╔══██╗██║╚██╗██║ ╚█████╔╝╚██████╔╝███████╗██║ ██║██║ ╚████║ ╚════╝ ╚═════╝ ╚══════╝╚═╝ ╚═╝╚═╝ ╚═══╝ Version: 2.0.1 Type `help` to begin joern> ``` -------------------------------- ### Interactive Joern Installation Source: https://github.com/joernio/joern/blob/master/README.md Use the interactive installation script if the standard installation script fails. This method allows for more user input during setup. ```bash ./joern-install --interactive ``` -------------------------------- ### Install and Run QueryDB Source: https://github.com/joernio/joern/blob/master/README.md Steps to stage Joern, install QueryDB, and list available queries. Add your own queries to the querydb directory and re-run these commands to deploy them. ```bash sbt stage ./querydb-install.sh ./joern-scan --list-query-names ``` -------------------------------- ### Install Joern and Query Database Source: https://github.com/joernio/joern/blob/master/querydb/README.md Installs Joern and the query database as an extension. This script is used for setting up the environment. ```bash ./install.sh ``` -------------------------------- ### Download and Setup Joern CLI Source: https://github.com/joernio/joern/blob/master/joern-cli/frontends/jimple2cpg/README.md Download the latest Joern CLI release and unzip it. Navigate into the Joern directory. ```bash wget https://github.com/joernio/joern/releases/latest/download/joern-cli.zip unzip joern-cli.zip cd joern-cli ``` -------------------------------- ### Install Dependencies for Swift on Ubuntu Source: https://github.com/joernio/joern/blob/master/joern-cli/frontends/swiftsrc2cpg/README.md Install necessary development packages on Ubuntu for Swift toolchain compatibility. This is a prerequisite for manual Swift installation. ```shell $ apt-get install \ binutils \ git \ gnupg2 \ libc6-dev \ libcurl4 \ libedit2 \ libgcc-9-dev \ libpython2.7 \ libsqlite3-0 \ libstdc++-9-dev \ libxml2 \ libz3-dev \ pkg-config \ tzdata \ uuid-dev \ zlib1g-dev ``` -------------------------------- ### Example: Scan a Directory with a C Function Source: https://github.com/joernio/joern/blob/master/querydb/README.md Demonstrates running `joern-scan` on a sample C file to identify functions with too many parameters. ```bash mkdir foo echo "int foo(int a, int b, int c, int d, int e, int f) {}" > foo/foo.c ./joern-scan foo ``` -------------------------------- ### Install PHP Source: https://github.com/joernio/joern/blob/master/joern-cli/frontends/php2cpg/README.md Install PHP version 7.0 or higher on your system. This is a prerequisite for the PHP to CPG converter. ```bash sudo apt install php ``` -------------------------------- ### Old Semantic Notation Examples Source: https://github.com/joernio/joern/blob/master/changelog/passthrough_semantics.md Illustrates the previous way of defining method parameter mappings, including explicit index-to-index and name-to-index mappings. ```text "method" 1 -> 2, "foo" -> 2, 1 -> "bar", "foo" -> "bar" ".tupleLiteral" 1 -> 1, 2 -> 2, 3 -> 3, 4 -> 4, 1 -> -1, 2 -> -1, 3 -> -1, 4 -> -1 // etc. ``` -------------------------------- ### Execute a Joern Script Source: https://github.com/joernio/joern/blob/master/joern-cli/src/main/resources/scripts/README.md Example of how to run a script named 'my-script.sc' with a parameter using the `runScript` method on a CPG object. ```scala cpg.runScript("my-script.sc", Map("param" -> "value")) ``` -------------------------------- ### Access Path Tracking Example Source: https://github.com/joernio/joern/blob/master/semanticcpg/src/main/scala/io/shiftleft/semanticcpg/accesspath/AccessPathAlgebra.md Illustrates access path tracking within a C++ function, showing policy entry and exit conditions, and access path (AP) and control path (CP) computations for different arguments. ```cpp void foo(T1 arg1, T2 arg2){ arg1[1] = *arg2; //return from policy with arg2: <0> * //enter with LHS, continue with arg1: <3> \ <-3> *, <-2> * //AP arg1: <3> <-3> <1> *, arg2: <0> * //CP arg1: <3> \ <-3> * } ``` -------------------------------- ### New Semantic Notation Examples Source: https://github.com/joernio/joern/blob/master/changelog/passthrough_semantics.md Demonstrates the updated semantic notation using named arguments and the `PASSTHROUGH` keyword for simplified flow definitions. ```text "method" 1 "foo" -> 2 "bar" ".tupleLiteral" PASSTHROUGH "setProperty" PASSTHROUGH 0 -> 0 ``` -------------------------------- ### Run Integration Tests Locally Source: https://github.com/joernio/joern/blob/master/README.md Execute integration tests by staging Joern, creating a distribution, installing necessary Python packages, and running the test script. This requires Python and the 'requests' and 'pexpect' libraries. ```bash sbt joerncli/stage querydb/createDistribution python -m pip install requests pexpect # wexpect on Windows python -u ./testDistro.py ``` -------------------------------- ### Run Joern in Server Mode with Docker Source: https://github.com/joernio/joern/blob/master/README.md Start Joern in server mode using a Docker container. This is useful for running Joern as a long-running service. ```bash docker run --rm -it -v /tmp:/tmp -v $(pwd):/app:rw -w /app -t ghcr.io/joernio/joern joern --server ``` -------------------------------- ### Adding dependencies via command line Source: https://github.com/joernio/joern/blob/master/changelog/2.0.0-scala3.md Specifies dependencies using the `--dep` parameter on the command line. This parameter can be used multiple times to add several dependencies. All dependencies must be declared when Joern starts. ```bash ./joern --dep com.michaelpollmeier:versionsort:1.0.7 ``` -------------------------------- ### Run c2cpg to generate CPG Source: https://github.com/joernio/joern/blob/master/joern-cli/frontends/c2cpg/README.md Generate a code property graph by running the c2cpg script with the source code directory and an output file path. Ensure Java runtime 11 and sbt are installed. ```bash ./c2cpg.sh --output ``` -------------------------------- ### Define a Query for Too Many Parameters Source: https://github.com/joernio/joern/blob/master/querydb/README.md An example of defining a custom query using Scala. This query identifies functions with more than a specified number of parameters. ```scala object Metrics extends QueryBundle { @q def tooManyParameters(n: Int = 4): Query = Query.make( name = "too-many-params", author = Crew.fabs, title = s"Number of parameters larger than $n", description = s"This query identifies functions with more than $n formal parameters", score = 1.0, withStrRep({ cpg => cpg.method.internal.filter(_.parameter.size > n) }), tags = List(QueryTags.metrics) ) @q def tooHighComplexity(n: Int = 4): Query = Query.make( name = "too-high-complexity", author = Crew.fabs, title = s"Cyclomatic complexity higher than $n", description = s"This query identifies functions with a cyclomatic complexity higher than $n", score = 1.0, withStrRep({ cpg => cpg.method.internal.filter(_.controlStructure.size > n) }), tags = List(QueryTags.metrics) ) ... } ``` -------------------------------- ### Build JavaScript AST Generator Source: https://github.com/joernio/joern/blob/master/joern-cli/frontends/jssrc2cpg/README.md Clone the astgen-monorepo and install dependencies to build native binaries for JavaScript AST generation. These binaries are required by jssrc2cpg. ```shell git clone https://github.com/joernio/astgen-monorepo.git cd astgen-monorepo/javascript-astgen yarn install ``` -------------------------------- ### Unit Test for Too Many Parameters Query Source: https://github.com/joernio/joern/blob/master/querydb/README.md Example of a unit test for the `tooManyParameters` query. It verifies that the query correctly identifies functions with an excessive number of parameters. ```scala class MetricsTests extends Suite { override val code = """ int too_many_params(int a, int b, int c, int d, int e) { } ... """ "find functions with too many parameters" in { Metrics.tooManyParameters(4)(cpg).map(_.evidence) match { case List(List(method: nodes.Method)) => method.name shouldBe "too_many_params" case _ => fail } } ... } ``` -------------------------------- ### Build and Run a Frontend Source: https://github.com/joernio/joern/blob/master/bazel.md Build and execute a specific frontend, passing arguments to it. Useful for development cycles. ```bash bazel run -- ``` ```bash bazel run javasrc2cpg -- /tmp/someJavaCodebase -o /tmp/myJava.cpg ``` -------------------------------- ### Main entry point in Scala 3 Source: https://github.com/joernio/joern/blob/master/changelog/2.0.0-scala3.md Shows the recommended way to define a main entry point in Scala 3, using `def main(args: Array[String])` within an object. The `extends App` pattern may lead to `NullPointerException` due to changes in Scala 3's handling of `DelayedInit`. ```scala object Main extends App { println("hello world") } // depending on usage, may lead to NullPointerExceptions // context: Scala3 doesn't support the 'magic' DelayedInit trait // rewrite to: object Main { def main(args: Array[String]) = { println("hello world") } } ``` -------------------------------- ### Build jimple2cpg Source: https://github.com/joernio/joern/blob/master/joern-cli/frontends/jimple2cpg/README.md Build the project using sbt. This command stages the project for execution. ```bash sbt stage ``` -------------------------------- ### Display c2cpg help and options Source: https://github.com/joernio/joern/blob/master/joern-cli/frontends/c2cpg/README.md View a comprehensive list of available command-line options for the c2cpg script by running it with the --help flag. ```bash ./c2cpg.sh --help ``` -------------------------------- ### Configure Bazel Cache Settings Source: https://github.com/joernio/joern/blob/master/bazel.md Set up the local Bazel cache location and maximum size in the ~/.bazelrc file. ```bash common --disk_cache ~/.cache/bazelBuildCache common --experimental_disk_cache_gc_max_size=30G ``` -------------------------------- ### Provide Swift Build Log Source: https://github.com/joernio/joern/blob/master/joern-cli/frontends/swiftsrc2cpg/README.md For projects not using SwiftPM, capture the build output to a file and provide it to swiftsrc2cpg using the --build-log-path option. This enables the extraction of type information. ```shell --build-log-path ``` -------------------------------- ### Run Formatting with Scalafmt Source: https://github.com/joernio/joern/blob/master/bazel.md Execute the scalafmt code formatter using Bazel. ```bash bazel run format ``` -------------------------------- ### Display Help for swiftsrc2cpg Source: https://github.com/joernio/joern/blob/master/joern-cli/frontends/swiftsrc2cpg/README.md Run the swiftsrc2cpg script with the --help flag to view all available command-line options and their descriptions. This is useful for understanding the full range of customization available. ```shell > ./swiftsrc2cpg.sh --help ``` -------------------------------- ### Flow Tracking with Function Calls Source: https://github.com/joernio/joern/blob/master/semanticcpg/src/main/scala/io/shiftleft/semanticcpg/accesspath/AccessPathAlgebra.md Demonstrates tracking data flow through function calls, including scenarios with pointer arithmetic and array indexing. Shows how the access path changes upon entering and exiting function calls. ```cpp q = source(); //find flow via q * foo(p+2, q); //return twice: [q: *] and [p: <2> <3> \ <-3> *, <-2> * //enter with arg1: <3> \ <-3> * //AP p: <2> //CP p: <2> <-2> <5> \ <-3> * ``` ```cpp foo(p[2], q); //would enter with arg1: <3> * which is excluded //AP p <5> <-5> <2> * //CP p <5> \ <3> * ``` ```cpp p[2] = 0; // track: p: <5> \ <-3> * ``` ```cpp sink(p+5); // track p: <5> ``` -------------------------------- ### Build and Test Joern Query Database Source: https://github.com/joernio/joern/blob/master/querydb/README.md Builds and tests the Joern Query Database using sbt. Ensures the database and its queries are functioning correctly. ```bash sbt test ``` -------------------------------- ### Check Formatting with Scalafmt Source: https://github.com/joernio/joern/blob/master/bazel.md Verify code formatting using scalafmt via Bazel. ```bash bazel run formatCheck ``` -------------------------------- ### Run c2cpg with additional options Source: https://github.com/joernio/joern/blob/master/joern-cli/frontends/c2cpg/README.md Extend CPG generation by specifying include paths, preprocessor definitions, and other options. Use comma-separated paths for multiple include directories. ```bash ./c2cpg.sh \ --output \ --include , \ --define DEF \ --define DEF_VAL=2 ``` -------------------------------- ### Build All Root Project Code Source: https://github.com/joernio/joern/blob/master/bazel.md Compile all code targets associated with the root of the Joern project using Bazel. ```bash bazel build //... ``` -------------------------------- ### Build and Generate CPG with abap2cpg Source: https://github.com/joernio/joern/blob/master/joern-cli/frontends/abap2cpg/README.md Follow these steps to build the abap2cpg frontend and generate a CPG from ABAP sources. The frontend binaries are downloaded automatically during the build process. ```bash # 1. Build the frontend (abapgen binaries are downloaded from astgen-monorepo by build.sbt) sbt abap2cpg/stage ``` ```bash # 2. Generate CPG ./abap2cpg.sh /path/to/abap/sources -o cpg.bin ``` ```bash # 3. Analyze in Joern joern joern> importCpg("cpg.bin") joern> cpg.call.name("AUTHORITY_CHECK").l // Find authorization checks joern> cpg.call.name(".*EXEC.*").l // Find potential command injection ``` -------------------------------- ### Run All Root Project Tests Source: https://github.com/joernio/joern/blob/master/bazel.md Execute all test targets within the root of the Joern project using Bazel. ```bash bazel test //... ``` -------------------------------- ### Basic Data Flow Traversal Source: https://github.com/joernio/joern/blob/master/dataflowengineoss/README.md Demonstrates basic usage for traversing data flow in the backwards direction. Assumes Joern shell environment with pre-configured imports and engine context. ```scala // If using Joern shell, imports and engine context will be pre-configured and available already import io.shiftleft.semanticcpg.language._ import io.joern.dataflowengineoss.language.toExtendedCfgNode def sink = cpg.call.argument.code(".*malicious_input.*") def source = cpg.call(".*println.*") // Traverses data flow in the backwards direction sink.reachableBy(source) ``` -------------------------------- ### Enable Swift Compiler Type Information Source: https://github.com/joernio/joern/blob/master/joern-cli/frontends/swiftsrc2cpg/README.md Use the --swift-build flag to enable extraction of full type information by invoking the Swift compiler. This requires Swift version 6.1 or later. ```shell ./swiftsrc2cpg.sh --swift-build ``` -------------------------------- ### c2cpg CLI Arguments Help Source: https://github.com/joernio/joern/blob/master/joern-cli/frontends/c2cpg/README.md This output details the command-line arguments for the C2Cpg tool, including input directory specification, output file naming, exclusion patterns, and various enabling/disabling flags for features like schema checking, file content inclusion, and comment logging. ```text Usage: C2Cpg [options] input-dir input-dir source directory -o, --output output filename --exclude files or folders to exclude during CPG generation (paths relative to or absolute paths) --exclude-regex a regex specifying files to exclude during CPG generation (paths relative to are matched) --enable-early-schema-checking enables early schema validation during AST creation (disabled by default) --enable-file-content add the raw source code to the content field of FILE nodes to allow for method source retrieval via offset fields (disabled by default) --help display this help message --include-comments includes all comments into the CPG --log-problems enables logging of all parse problems while generating the CPG --log-preprocessor enables logging of all preprocessor statements while generating the CPG --print-ifdef-only prints a comma-separated list of all preprocessor ifdef and if statements; does not create a CPG --include header include paths --with-include-auto-discovery enables auto discovery of system header include paths --skip-function-bodies instructs the parser to skip function and method bodies. --with-preprocessed-files includes *.i files and gives them priority over their unprocessed origin source files. --define define a name --compilation-database enables the processing of compilation database files (e.g., compile_commands.json). This allows to automatically extract compiler options, source files, and other build information from the specified database and ensuring consistency with the build configuration. For a cmake based build such a file is generated with the environment variable CMAKE_EXPORT_COMPILE_COMMANDS being present. Clang based build are supported e.g., with https://github.com/rizsotto/Bear ``` -------------------------------- ### Run All Queries on a Code Directory Source: https://github.com/joernio/joern/blob/master/querydb/README.md Executes all available queries in the Joern Query Database against the specified code path. Useful for quick code analysis. ```bash ./joern-scan path/to/code ``` -------------------------------- ### Import Java Code into Joern Source: https://github.com/joernio/joern/blob/master/joern-cli/frontends/javasrc2cpg/README.md Use this command to import your Java source code into Joern after setting up the joern-cli. Ensure you replace `` with the actual path to your Java project. ```bash importCode.javasrc("") ``` -------------------------------- ### Run Joern with Docker on Almalinux 8 Source: https://github.com/joernio/joern/blob/master/README.md Use the Almalinux 8 Docker image for Joern if your CPU does not support SSE4.2 or if you are using a kvm64 VM. This ensures compatibility with older environments. ```bash docker run --rm -it -v /tmp:/tmp -v $(pwd):/app:rw -w /app -t ghcr.io/joernio/joern-alma8 joern ``` -------------------------------- ### Run All abap2cpg Tests Source: https://github.com/joernio/joern/blob/master/joern-cli/frontends/abap2cpg/architecture.md Execute all tests for the abap2cpg module, which requires the abapgen binary to be available. ```bash sbt "abap2cpg/test" ``` -------------------------------- ### jssrc2cpg Help and Options Source: https://github.com/joernio/joern/blob/master/joern-cli/frontends/jssrc2cpg/README.md Display the help message for jssrc2cpg to view all available options for CPG generation. This includes options for output, exclusions, and schema checking. ```shell > ./jssrc2cpg.sh --help Usage: jssrc2cpg [options] input-dir input-dir source directory -o, --output output filename --exclude files or folders to exclude during CPG generation (paths relative to or absolute paths) --exclude-regex a regex specifying files to exclude during CPG generation (paths relative to are matched) --enable-early-schema-checking enables early schema validation during AST creation (disabled by default) --enable-file-content add the raw source code to the content field of FILE nodes to allow for method source retrieval via offset fields (disabled by default) --help display this help message ``` -------------------------------- ### Explicit Traversal Conversion with .start Source: https://github.com/joernio/joern/blob/master/changelog/traversal_removal.md Alternatively, use the `.start` method to explicitly convert a node into a `Traversal` before applying traversal methods. This is an alternative to `Iterator(element)`. ```scala def globalFromLiteral(lit: Literal): Traversal[Expression] = lit.start .where(_.inAssignment.method.nameExact("", ":package")) .inAssignment .argument(1) ``` -------------------------------- ### Build Swift AST Generator Source: https://github.com/joernio/joern/blob/master/joern-cli/frontends/swiftsrc2cpg/README.md Clone the astgen-monorepo and navigate to the swift-astgen directory to build native binaries for Swift AST generation. This requires the 'swift' toolchain. ```shell git clone https://github.com/joernio/astgen-monorepo.git cd astgen-monorepo/swift-astgen swift build ``` -------------------------------- ### Stage and Test Custom Queries with Joern-Scan Source: https://github.com/joernio/joern/blob/master/querydb/README.md Stages the Joern CLI and then runs `joern-scan` to test newly developed queries. This is part of the development workflow for custom queries. ```bash sbt joerncli/stage ./querydb-install.sh && ./joern-scan ``` -------------------------------- ### Create CPG with jimple2cpg Source: https://github.com/joernio/joern/blob/master/joern-cli/frontends/jimple2cpg/README.md Generate a CPG from Jimple code using the jimple2cpg script. Specify input code path and output CPG file path. ```bash ./jimple2cpg.sh /path/to/your/code -o /path/to/cpg.bin ``` -------------------------------- ### Run Joern with Docker Source: https://github.com/joernio/joern/blob/master/README.md Execute Joern within a Docker container, mounting local directories for temporary files, application code, and working directory. This provides a consistent execution environment. ```bash docker run --rm -it -v /tmp:/tmp -v $(pwd):/app:rw -w /app -t ghcr.io/joernio/joern joern ``` -------------------------------- ### Dump Available Queries to JSON Source: https://github.com/joernio/joern/blob/master/querydb/README.md Exports a list of all available queries and their metadata to a JSON file named `querydb.json`. This is useful for reviewing or integrating query information. ```bash ./joern-scan --dump ``` -------------------------------- ### Run abap2cpg Unit Tests Source: https://github.com/joernio/joern/blob/master/joern-cli/frontends/abap2cpg/architecture.md Execute only the unit tests for the abap2cpg module. This command does not require the abapgen binary. ```bash sbt "abap2cpg/testOnly *.AstCreatorTests" ``` -------------------------------- ### Modern Joern Query Style (Generated APIs) Source: https://github.com/joernio/joern/blob/master/changelog/traversal_removal.md Demonstrates the preferred, modern Joern API style using generated domain-specific methods (e.g., `_astOut`) for accessing graph elements. This approach is more performant and memory-efficient than string-based lookups. ```scala def topLevelExpressions: Traversal[Expression] = traversal ._astOut .collectAll[Block] ._astOut .not(_._collectAll[Local]) ``` -------------------------------- ### Passing script parameters in Joern Source: https://github.com/joernio/joern/blob/master/changelog/2.0.0-scala3.md Illustrates the new method for passing script parameters, using multiple `--param` flags instead of a single comma-separated list. This change improves readability and allows parameter values to contain commas. ```bash // old ./joern --script foo.sc --params paramA=valueA,paramB=valueB // new ./joern --script foo.sc --param paramA=valueA --param paramB=valueB ``` -------------------------------- ### Import CPG into Joern Source: https://github.com/joernio/joern/blob/master/joern-cli/frontends/jimple2cpg/README.md After copying the generated CPG file into the Joern directory, import it using the importCpg command within the Joern shell. ```scala importCpg("cpg.bin") ``` -------------------------------- ### Data Flow Engine Configuration Source: https://github.com/joernio/joern/blob/master/dataflowengineoss/README.md Provides necessary imports and configuration for using the data flow engine on a CPG. Includes optional engine configuration and creation of an execution context. ```scala // (1) Imports to extend CFG nodes import io.joern.dataflowengineoss.language.toExtendedCfgNode import io.joern.dataflowengineoss.queryengine.{EngineContext, EngineConfig} import scala.util.{Failure, Success, Try} import scala.io.{BufferedSource, Source} // (2) Optional: Configure the engine val engineConfig = EngineConfig(maxCallDepth = 2, initialTable = None, disableCacheUse = false) // (3) Create execution context for the engine implicit var context: EngineContext = EngineContext(config = engineConfig) ``` -------------------------------- ### Generate CPG with Define Options Source: https://github.com/joernio/joern/blob/master/joern-cli/frontends/swiftsrc2cpg/README.md Generate a code property graph with additional options, including defining custom values. This allows for conditional compilation or configuration during CPG generation. ```shell ./swiftsrc2cpg.sh \ --output \ --define DEF --define DEF_VAL=2 ``` -------------------------------- ### Historical Joern Query Style (String-based) Source: https://github.com/joernio/joern/blob/master/changelog/traversal_removal.md Illustrates the older, property-graph-database-style query building using string-based lookups for edges and node labels. This approach is less efficient and memory-intensive. ```scala def topLevelExpressions: Traversal[Expression] = traversal .out(EdgeTypes.AST) .hasLabel(NodeTypes.BLOCK) .out(EdgeTypes.AST) .not(_.hasLabel(NodeTypes.LOCAL)) ``` -------------------------------- ### List Available Joern Scripts Source: https://github.com/joernio/joern/blob/master/joern-cli/src/main/resources/scripts/README.md This method returns a list of available scripts, including their names and descriptions. ```scala def scripts(): List[ScriptDescription] ``` -------------------------------- ### Generate JSON AST with php-parse Source: https://github.com/joernio/joern/blob/master/joern-cli/frontends/php2cpg/README.md Invoke the `php-parse` tool with the `--json-dump` and `--with-recovery` flags for each PHP source file. This generates an Abstract Syntax Tree (AST) in JSON format. ```bash php-parse --json-dump --with-recovery $file ``` -------------------------------- ### Generate Code Property Graph Source: https://github.com/joernio/joern/blob/master/joern-cli/frontends/swiftsrc2cpg/README.md Execute the swiftsrc2cpg script to generate a code property graph from a Swift source code directory. Specify input and output paths. ```shell ./swiftsrc2cpg.sh --output ``` -------------------------------- ### Using file directive in Joern scripts Source: https://github.com/joernio/joern/blob/master/changelog/2.0.0-scala3.md Replaces the '$file.foo' magic from Ammonite with the '//> using file foo.sc' directive for including script files. This works in both active Joern REPL sessions and standalone scripts. ```scala //> using file foo.sc ``` -------------------------------- ### Adding dependencies within a script Source: https://github.com/joernio/joern/blob/master/changelog/2.0.0-scala3.md Declares dependencies directly within a script using the '//> using dep' directive. This provides a cleaner alternative to command-line dependency specification for scripts. Dependencies must be known at Joern startup. ```scala //> using dep com.michaelpollmeier:versionsort:1.0.7 ``` -------------------------------- ### General Joern Slice Options Source: https://github.com/joernio/joern/blob/master/joern-cli/JOERN_SLICE.md These options are shared across all Joern Slice modes. They control input CPG files, output destinations, and filtering. ```text cpg input CPG file name, or source code - defaults to `cpg.bin` -o, --out the output file to write slices to - defaults to `slices`. The file is suffixed based on the mode. --dummy-types for generating CPGs that use type recovery, enables the use of dummy types - defaults to false. --file-filter the name of the source file to generate slices from. --method-name-filter filters in slices that go through specific methods by names. Uses regex. --method-parameter-filter filters in slices that go through methods with specific types on the method parameters. Uses regex. --method-annotation-filter filters in slices that go through methods with specific annotations on the methods. Uses regex. ``` -------------------------------- ### Usages Slice Options Source: https://github.com/joernio/joern/blob/master/joern-cli/JOERN_SLICE.md Options specific to the Usages slicing mode. This mode tracks how variables interact within their procedure. ```text Command: usages [options] --min-num-calls the minimum number of calls required for a usage slice - defaults to 1. --exclude-operators excludes operator calls in the slices - defaults to false. --exclude-source excludes method source code in the slices - defaults to false. ``` -------------------------------- ### Define `runScript` Overloads Source: https://github.com/joernio/joern/blob/master/joern-cli/src/main/resources/scripts/README.md These are the available overloads for the `runScript` method in Joern. ```scala def runScript(name: String, params: Map[String, String]): AnyRef def runScript(name: String, params: Map[String, String], cpg: Cpg): AnyRef def runScript(name: String, params: Map[String, String], cpgFilename: String): AnyRef ``` -------------------------------- ### Data-Flow Slice Options Source: https://github.com/joernio/joern/blob/master/joern-cli/JOERN_SLICE.md Options specific to the Data-flow slicing mode. This mode performs interprocedural backward slicing. ```text Command: data-flow [options] --slice-depth the max depth to traverse the DDG for the data-flow slice - defaults to 20. --sink-filter filters on the sink's `code` property. Uses regex. --end-at-external-method all slices must end at an external method - defaults to false. ``` -------------------------------- ### Using CPG API for TaggedBy Traversal Source: https://github.com/joernio/joern/blob/master/changelog/traversal_removal.md Prefer using the domain-specific CPG API, such as `_taggedByIn`, for more idiomatic and potentially efficient traversal operations. ```scala private def tagged[A <: StoredNode: ClassTag]: Traversal[A] = traversal._taggedByIn.collectAll[A].sortBy(_.id).iterator ``` -------------------------------- ### Adding Traversal Extension Imports Source: https://github.com/joernio/joern/blob/master/changelog/traversal_removal.md Ensures `import io.shiftleft.semanticcpg.language._` is present to enable operations on Traversal that were previously implemented as member methods. This is necessary for certain traversal extensions to function correctly. ```scala [error] joern/semanticcpg/src/main/scala/io/shiftleft/semanticcpg/language/types/structure/AnnotationParameterAssignTraversal.scala:21:8: value cast is not a member of Iterator[io.shiftleft.codepropertygraph.generated.nodes.AstNode] [error] did you mean wait? [error] possible cause: maybe a semicolon is missing before `value cast`? [error] .cast[Expression] ``` -------------------------------- ### Explicit Iterator Conversion for Extension Methods Source: https://github.com/joernio/joern/blob/master/changelog/traversal_removal.md When defining extension methods that accept `IterableOnce`, explicitly convert the input to an `Iterator` using `.iterator` before constructing the target type. This ensures consistent handling of traversals. ```scala implicit def toDdgNodeDot(traversal: IterableOnce[Method]): DdgNodeDot = new DdgNodeDot(traversal.iterator) ``` -------------------------------- ### MethodUsageSlice Scala Case Class Source: https://github.com/joernio/joern/blob/master/joern-cli/JOERN_SLICE.md Represents usage slices within a specific method, including code, location, and object usage details. ```scala case class MethodUsageSlice( code: String, fullName: String, fileName: String, slices: Set[ObjectUsageSlice], lineNumber: Option[Int] = None, columnNumber: Option[Int] = None ) ``` -------------------------------- ### Generate CPG from JavaScript/TypeScript Source: https://github.com/joernio/joern/blob/master/joern-cli/frontends/jssrc2cpg/README.md Use the jssrc2cpg.sh script to generate a code property graph (CPG) from a given source code directory. Specify the output file for the CPG. ```shell ./jssrc2cpg.sh --output ``` -------------------------------- ### UserDefinedType Scala Case Class Source: https://github.com/joernio/joern/blob/master/joern-cli/JOERN_SLICE.md Represents a user-defined type, including its fields, procedures, and source file information. ```scala case class UserDefinedType( name: String, fields: List[LocalDef], procedures: List[ObservedCall], fileName: String = "", lineNumber: Option[Int] = None, columnNumber: Option[Int] = None ) ``` -------------------------------- ### Replace PathAwareTraversal Companion Object Source: https://github.com/joernio/joern/blob/master/changelog/traversal_removal.md Replace calls to the `PathAwareTraversal` companion object with `Iterator` or `Iterator.empty` as appropriate. The old `PathAwareTraversal` companion object is no longer available. ```scala [error] joern/semanticcpg/src/main/scala/io/shiftleft/semanticcpg/language/callgraphextension/MethodTraversal.scala:18:7: not found: value PathAwareTraversal [error] PathAwareTraversal.empty ``` -------------------------------- ### abap2cpg Module Structure Source: https://github.com/joernio/joern/blob/master/joern-cli/frontends/abap2cpg/README.md The codebase is organized into distinct modules for parsing, AST creation, and CPG passes. This structure facilitates a modular approach to code analysis. ```tree abap2cpg/ ├── parser/ │ ├── AbapAstGenRunner.scala # Invokes Node.js parser │ ├── AbapJsonParser.scala # JSON → Intermediate AST │ └── AbapIntermediateAst.scala # AST data structures │ ├── astcreation/ │ ├── AstCreator.scala # Main orchestrator (126 lines) │ ├── AstHelpers.scala # Code generation utilities │ │ │ ├── declarations/ │ │ ├── AstForDeclarationsCreator.scala # Methods, types │ │ └── AstForParametersCreator.scala # Parameters, locals │ │ │ ├── expressions/ │ │ ├── AstForExpressionsCreator.scala # Expression dispatcher │ │ ├── AstForCallsCreator.scala # Call expressions │ │ └── AstForSimpleExpressionsCreator.scala # Identifiers, literals │ │ │ └── statements/ │ └── AstForStatementsCreator.scala # Assignments, declarations │ └── passes/ ├── AstCreationPass.scala # CPG creation pass ├── RefEdgePass.scala # Variable reference edges └── NamespacePass.scala # Namespace creation ``` -------------------------------- ### ObjectUsageSlice Scala Case Class Source: https://github.com/joernio/joern/blob/master/joern-cli/JOERN_SLICE.md Details the usage of an object within a method, including where it's defined and the calls it makes or is part of. ```scala case class ObjectUsageSlice( targetObj: DefComponent, definedBy: Option[DefComponent], invokedCalls: List[ObservedCall], argToCalls: List[ObservedCallWithArgPos] ) ``` -------------------------------- ### ProgramUsageSlice Scala Case Class Source: https://github.com/joernio/joern/blob/master/joern-cli/JOERN_SLICE.md Defines the overall structure for a program usage slice, containing method usage slices and user-defined types. ```scala case class ProgramUsageSlice(objectSlices: List[MethodUsageSlice], userDefinedTypes: List[UserDefinedType]) ``` -------------------------------- ### Fixing Ambiguous Traversal Import Source: https://github.com/joernio/joern/blob/master/changelog/traversal_removal.md Removes the redundant `import overflowdb.traversal._` to resolve ambiguity when `io.shiftleft.semanticcpg.language._` is also imported. This was a common issue affecting multiple files. ```scala [error] joern/semanticcpg/src/main/scala/io/shiftleft/semanticcpg/language/nodemethods/StoredNodeMethods.scala:11:12: reference to Traversal is ambiguous; [error] it is imported twice in the same scope by [error] import overflowdb.traversal._ [error] and import io.shiftleft.semanticcpg.language._ [error] def tag: Traversal[Tag] = { ``` -------------------------------- ### Replace Traversal Companion Object with Iterator Source: https://github.com/joernio/joern/blob/master/changelog/traversal_removal.md Replace calls to the `Traversal` companion object factory with the `Iterator` companion object factory. The old `Traversal` companion object is no longer available. ```scala def dotCallGraph(cpg: Cpg): Traversal[String] = { val callGraph = new CallGraphGenerator().generate(cpg) Traversal(DotSerializer.dotGraph(None, callGraph)) } ``` ```scala def dotCallGraph(cpg: Cpg): Traversal[String] = { val callGraph = new CallGraphGenerator().generate(cpg) Iterator(DotSerializer.dotGraph(None, callGraph)) } ``` -------------------------------- ### Anonymous function parameter syntax in Scala 3 Source: https://github.com/joernio/joern/blob/master/changelog/2.0.0-scala3.md Demonstrates the change in anonymous function syntax in Scala 3, where parentheses are required around the parameter list if a type annotation is present. If the type annotation can be omitted, parentheses are not needed. ```scala Seq(1,2,3).map { i: Int => i + 1 } // error: parentheses are required around the parameter of a lambda // option 1: add parentheses, as suggested by compiler: Seq(1,2,3).map { (i: Int) => i + 1 } // option 2: drop type annotation (if possible): Seq(1,2,3).map { i => i + 1 } ``` -------------------------------- ### Explicit Traversal Conversion with Iterator(lit) Source: https://github.com/joernio/joern/blob/master/changelog/traversal_removal.md To explicitly convert a single element into a `Traversal`, wrap it using `Iterator(element)`. This is required when methods like `where` are called on non-traversal types. ```scala def globalFromLiteral(lit: Literal): Traversal[Expression] = Iterator(lit) .where(_.inAssignment.method.nameExact("", ":package")) .inAssignment .argument(1) ``` -------------------------------- ### DataFlowSlice Scala Case Class Source: https://github.com/joernio/joern/blob/master/joern-cli/JOERN_SLICE.md Defines the structure for a data-flow slice, including nodes and edges. ```scala case class DataFlowSlice(nodes: Set[SliceNode], edges: Set[SliceEdge]) ``` -------------------------------- ### ObservedCall Scala Case Class Source: https://github.com/joernio/joern/blob/master/joern-cli/JOERN_SLICE.md Represents a call made by an object, including its name, resolved method, parameter types, and return type. ```scala case class ObservedCall( callName: String, resolvedMethod: Option[String], paramTypes: List[String], returnType: String, lineNumber: Option[Int] = None, columnNumber: Option[Int] = None ) ``` -------------------------------- ### SliceNode Scala Case Class Source: https://github.com/joernio/joern/blob/master/joern-cli/JOERN_SLICE.md Represents a node within a data-flow slice, containing details about its code, type, and location. ```scala case class SliceNode( id: Long, label: String, name: String = "", code: String, typeFullName: String = "", parentMethod: String = "", parentFile: String = "", lineNumber: Option[Integer] = None, columnNumber: Option[Integer] = None ) ``` -------------------------------- ### Explicit Traversal Conversion with .iterator Source: https://github.com/joernio/joern/blob/master/changelog/traversal_removal.md When a `Traversal` is expected, explicitly convert sequences or iterables using `.iterator` to satisfy type requirements. This is necessary after the removal of implicit conversions. ```scala private def tagged[A <: StoredNode: ClassTag]: Traversal[A] = traversal.in(EdgeTypes.TAGGED_BY).collectAll[A].sortBy(_.id).iterator ```