SAS Event Stream Processing CDC

https://documentation.sas.com/doc/en/espcdc/v_071/
Admin
Comprehensive documentation for SAS Event Stream Processing (ESP) including deployment guides, usage...
Tokens:662,197
Snippets:8,607
Trust Score:10
Update:5 months ago
Show doc for...
Tokens:
Raw
### Start ESP Server with Token Argument

Source: https://documentation.sas.com/doc/en/espcdc/v_071/espxmllang/n0t0o9h7tpxco9n1gbgae9tsnir6.htm

Example of starting the ESP server and defining a token value for the INDEXTYPE. This allows dynamic configuration of window indexes.

```bash
$ dfesp_xml_server _arguments_ -INDEXTYPE '_pi_HASH_'
```

--------------------------------

### Group Instructions for Deploying Projects to ESP Servers

Source: https://documentation.sas.com/doc/en/espcdc/v_071/esmjt/p1srxywx128vbwn11r4kxenxn0jn.htm

This example demonstrates how to use the 'group' element to deploy two different projects to separate sets of ESP servers. Each group defines the loading, starting, and connector startup instructions for a project, along with a server filter to specify the target servers.

```xml
<instructions> <group id="startgroup1" localization-id="startgroup1" server-filter="filter1"> <load-project id="load-project1" localization-id="load-project" project-id="{project1.id}" project-version="{project1.version}" running-project-name="{project1display}" start="false" /> <start-project id="start-project1" localization-id="start-project" running-project-name="{project1}" depends-on="load-project1" /> <start-connectors id="start-connectors1" localization-id="start-connectors" running-project-name="{project1}" depends-on="start-project1" /> </group> <group id="startgroup2" localization-id="startgroup2" server-filter="filter2"> <load-project id="load-project2" localization-id="load-project" project-id="{project2.id}" project-version="{project2.version}" running-project-name="{project2display}" start="false" /> <start-project id="start-project2" localization-id="start-project" running-project-name="{project2}" depends-on="load-project2" /> <start-connectors id="start-connectors2" localization-id="start-connectors" running-project-name="{project2}" depends-on="start-project2" /> </group> </instructions>
```

```xml
<instructions>
   <group id="startgroup1" localization-id="startgroup1" 
    server-filter="filter1">
      <load-project id="load-project1" 
       localization-id="load-project" 
       project-id="{project1.id}" 
       project-version="{project1.version}" 
       running-project-name="{project1display}" 
       start="false" />
      <start-project id="start-project1" 
       localization-id="start-project" 
       running-project-name="{project1}" 
       depends-on="load-project1" />
      <start-connectors id="start-connectors1" 
       localization-id="start-connectors" 
       running-project-name="{project1}"
       depends-on="start-project1" />
   </group>
   <group id="startgroup2" localization-id="startgroup2" 
    server-filter="filter2">
      <load-project id="load-project2" 
       localization-id="load-project" 
       project-id="{project2.id}" 
       project-version="{project2.version}" 
       running-project-name="{project2display}" 
       start="false" />
      <start-project id="start-project2" 
       localization-id="start-project" 
       running-project-name="{project2}" 
       depends-on="load-project2" />
      <start-connectors id="start-connectors2" 
       localization-id="start-connectors" 
       running-project-name="{project2}" 
       depends-on="start-project2" />
   </group>
</instructions>
```

--------------------------------

### Starting an Adapter

Source: https://documentation.sas.com/doc/en/espcdc/v_071/espcreatewindows/n0jhyg9r2wzobwn1vdyxkat99urx.htm

Use the `esp_adapter_start` function to start a previously created adapter. The function takes the adapter object returned by `esp_adapter_create` as a parameter. A successful start returns a code of `0`.

```APIDOC
## Starting an Adapter

Use the following function to start an adapter:
```lua
local response = esp_adapter_start(adapter)
```

The `_adapter_`parameter is the return value from the`esp_adapter_create` function. 
The response is a Lua object that contains a code and optionally a message field. If the adapter is not currently running, the server attempts to start it and returns a code of `0`. Otherwise, you get an error code of `-1`. 
If you have defined the `adapter_started` callback function, it is invoked when the process has successfully started.
```

--------------------------------

### Start ESP Server

Source: https://documentation.sas.com/doc/en/espcdc/v_071/espxmllayer/p127n8rp8fzp7in1v4q0du185y6h.htm

Run this command after setting the DFESP_HOME and PATH environment variables to start the ESP server.

```bash
dfesp_xml_server
```

--------------------------------

### Start ESP Server with a Model

Source: https://documentation.sas.com/doc/en/espcdc/v_071/espxmllayer/p18dkdtyyd9o91n1ddra2vzxqfee.htm

Use this command to start an ESP server with an initial model file.

```bash
dfesp_xml_server -http 62000 -pubsub 62001 -model file://model.xml  
```

--------------------------------

### Input Variables Configuration Example

Source: https://documentation.sas.com/doc/en/espcdc/v_071/espcreatewindows/p1e2lkph7tr1u0n1pan9g8mgyezi.htm

Example configuration for defining input variables for a custom window, specifying fields like 'trade_symbol', 'trade_price', and 'trade_quantity'.

```json
"inputVariables": {
    "desc" : "Optional description of the input variables",
    "fields" : [
        {
            "name": "trade_symbol",
            "desc": "Stock symbol"
        },
        {
            "name": "trade_price",
            "desc": "Trade price"
        },
        {
            "name": "trade_quantity",
            "desc": "Trade quantity"
        }
    ],
}

```

--------------------------------

### Input Event Example

Source: https://documentation.sas.com/doc/en/espcdc/v_071/espan/p1vrvurw1fy9x7n106om6b1xdbw6.htm

Example of a single input event.

```text
i,n,1,field1,field2,field3
```

--------------------------------

### Start a Project

Source: https://documentation.sas.com/doc/en/espcdc/v_071/espxmllayer/p145wur4og154cn103ik6c67dgkl.htm

This operation starts a specified project, enabling it to process events. This is essential for activating a deployed project.

```APIDOC
## PUT /eventStreamProcessing/v1/projects/{projectName}/state?value=running

### Description
Starts the specified project.

### Method
PUT

### Endpoint
`/eventStreamProcessing/v1/projects/{projectName}/state?value=running`

### Parameters
#### Path Parameters
- **projectName** (string) - Required - The name of the project to start.

### Response
#### Success Response (200)
- **response** (xml) - An XML structure indicating the success of the project start operation.

### Response Example
```xml
<response>
    <message>project 'project' successfully started</message>
</response>
```
```

--------------------------------

### Start File and Socket Adapter Subscriber with RabbitMQ

Source: https://documentation.sas.com/doc/en/espcdc/v_071/espfailover/n17di0m3crfx8on173hqw89dgzw0.htm

Use this command to start a file and socket adapter in subscriber mode, connecting to an ESP server via RabbitMQ. Ensure the subscriber starts before the publisher to capture all events.

```bash
$DFESP_HOME/bin/dfesp_fs_adapter -k sub -h "dfESP://host:port/tradesRMQ/ trades/Aggregate24HR?snapshot=false" -f trades1M_aggr24hr.csv -t csv -l rabbitmq

```

```bash
$DFESP_HOME/bin/dfesp_fs_adapter -k sub -h "dfESP://host:port/tradesRMQ/
trades/Aggregate24HR?snapshot=false" -f trades1M_aggr24hr.csv -t csv -l rabbitmq

```

--------------------------------

### Send Request Example

Source: https://documentation.sas.com/doc/en/espcdc/v_071/espan/n1gk6djwzlve2on1iesy99su5rsi.htm

Example of a 'send' request to deploy a model to a downstream window. Requires modelId and target window name.

```text
i,n,1,"action","send"
i,n,2,"modelId","_USER_SPECIFIED_MODEL_ID_"
i,n,3,"target","_USER_SPECIFIED_TARGET_"
i,n,4,,

```

--------------------------------

### Example: Load Command with dfesp_xml_client

Source: https://documentation.sas.com/doc/en/espcdc/v_071/espxmllayer/p1uyrkk0eal12gn1g0kiq1yr0rvh.htm

This example demonstrates subscribing to a specific window and sending a `<load/>` command from a file. The output shows the received events in XML format.

```xml
<load/>
```

```bash
dfesp_xml_client                -url "ws://edge01:26000/eventStreamProcessing/v1/subscribers/p/cq/brokerAlertsAggr?format=xml"                -data "file://load.xml"
```

```xml
<events page='0' pages='0'>
    <event>
        <brokerName>Joe</brokerName>
        <closeMarking>0</closeMarking>
        <frontRunningBuy>36</frontRunningBuy>
        <frontRunningSell>58</frontRunningSell>
        <openMarking>68</openMarking>
        <restrictedTrades>28</restrictedTrades>
        <total>190</total>
    </event>
    <event>
        <brokerName>Curt</brokerName>
        <closeMarking>8</closeMarking>
        <frontRunningBuy>41</frontRunningBuy>
        <frontRunningSell>46</frontRunningSell>
        <openMarking>84</openMarking>
        <restrictedTrades>48</restrictedTrades>
        <total>227</total>
    </event>
    <event>
        <brokerName>Steve</brokerName>
        <closeMarking>12</closeMarking>
        <frontRunningBuy>16</frontRunningBuy>
        <frontRunningSell>40</frontRunningSell>
        <openMarking>72</openMarking>
        <restrictedTrades>28</restrictedTrades>
        <total>168</total>
    </event>
    <event>
        <brokerName>Lisa</brokerName>
        <closeMarking>12</closeMarking>
        <frontRunningBuy>40</frontRunningBuy>
        <frontRunningSell>50</frontRunningSell>
        <openMarking>92</openMarking>
        <restrictedTrades>32</restrictedTrades>
        <total>226</total>
    </event>
    <event>
        <brokerName>Sally</brokerName>
        <closeMarking>12</closeMarking>
        <frontRunningBuy>20</frontRunningBuy>
        <frontRunningSell>42</frontRunningSell>
        <openMarking>60</openMarking>
        <restrictedTrades>44</restrictedTrades>
        <total>178</total>
    </event>
</events>
```

--------------------------------

### Get Project Metadata with PROJECTMETADATA

Source: https://documentation.sas.com/doc/en/espcdc/v_071/espcreatewindows/n09udz94wp70tvn1ort9k6ad72oz.htm

Use PROJECTMETADATA to retrieve specific metadata items for a given project. The example shows how to get the ID of the MAS module used for scoring.

```SAS
projectMetadata('scoreproj','mm_linked_module1')=1.2
```

--------------------------------

### Get ASCII Character using CHR

Source: https://documentation.sas.com/doc/en/espcdc/v_071/engelref/p0xqo0d0z0ue5cn14ay82x9zhkyd.htm

Use the CHR function with an ASCII code to retrieve the corresponding character. This example shows how to get the letter 'a' and also demonstrates the inverse ASC function.

```string
character_content = chr(97) // outputs the letter "a"
ascii_value = asc("a") // outputs 97
```

--------------------------------

### Initialize dfesp_audio_adapter

Source: https://documentation.sas.com/doc/en/espcdc/v_071/espca/n09daj097d0qcqn1slud2kcg3xj4.htm

Use this command to initialize the audio adapter. Configure it with key-value pairs for essential parameters like URL and device name. Values containing commas must be enclosed in escaped double quotes.

```bash
dfesp_audio_adapter -C _key1_=_val1_, _key2_=_val2_, _key3_=_val3_…
```

--------------------------------

### Function Usage Example

Source: https://documentation.sas.com/doc/en/espcdc/v_071/engelref/n1w06w1wjvul9en1xuyuuh5bn3u7.htm

Illustrates calling functions like `upper`, `print`, and `instr`. `instr` returns the starting position of a substring.

```SAS
string str
integer x 
str="Hello there"
// calls the upper function
if upper(str)=='HELLO THERE'
     // calls the print function
     call print("yes")
 
// x is set to 7 (position of word 'there')
x=instr(str,"there",1)
```

--------------------------------

### Start a Publisher Adapter

Source: https://documentation.sas.com/doc/en/espcdc/v_071/espcreatewindows/n0jhyg9r2wzobwn1vdyxkat99urx.htm

Use `esp_adapter_start` to initiate an adapter. The function takes the adapter object returned by `esp_adapter_create` as input. A return code of 0 indicates success.

```lua
local response = esp_adapter_start(_adapter_)
```

--------------------------------

### Using SUBSTRINGSTART with Regular Expressions

Source: https://documentation.sas.com/doc/en/espcdc/v_071/engelref/n0itbr9cv1tsyrn1f4jbulaukppi.htm

This example demonstrates how to use the SUBSTRINGSTART function to find the starting position of captured sub-patterns within a string. It compiles a regular expression with sub-patterns, iterates through the found substrings, and extracts their start and length to reconstruct the matched sub-pattern.

```regex
//Define some variables
string MyString
string MyString2
integer i
integer SSC
integer SSS
integer SSL

// Set initial values for variables
i = 0
SSS = 0
SSL = 0
SSC = 0

// Sample inpit string
MyString = "DataFlux Data Management Studio"

// Define a regular expression object
regex r
// Then compile it - notice the use of ( and )
r. compile("(DataFlux|DF) Data Management (Studio|Platform)")
// Find the first substring
if r.findfirst(MyString)
   begin
      // Use the "substring" functions to find the number of substrings
      SSC = r.substringcount()
      // Loop through substrings
      for i = 1 to SSC
      begin
      // Then pull out substrings
      SSS = r.substringstart(i)
      SSL = r.substringlength(i)
      MyString2 = mid(MyString,SSS,SSL)
      // Place the substrings in a data row
      pushrow()
      end
   end
// Terminate the expression node processing
seteof(true)
// Prevent the last pushrow() from showing up twice
return false
```

--------------------------------

### Output Variables Configuration Example

Source: https://documentation.sas.com/doc/en/espcdc/v_071/espcreatewindows/p1e2lkph7tr1u0n1pan9g8mgyezi.htm

Example configuration for defining output variables for a custom window, specifying fields like 'maxp' for maximum price and 'maxq' for maximum quantity.

```json
"outputVariables": {
    "desc" : "Optional description of the output variables",
    "fields" : [
        {
            "name": "maxp",
            "desc": "Maximum price"
        },
        {
            "name": "maxq",
            "desc": "Maximum quantity"
        }
    ],
}

```

--------------------------------

### Get Minute of the Day with timeMinuteOfDay

Source: https://documentation.sas.com/doc/en/espcdc/v_071/espcreatewindows/p1b9sd4ykolfkwn1orjfocca3xs3.htm

timeMinuteOfDay returns the total number of minutes from the start of the day for a given time. It defaults to the current time if no argument is supplied.

```espcdc
timeMinuteOfDay()=563
timeMinuteOfDay(timeParse('06/21/2015 13:45:15','%m/%d/%Y %H:%M:%S'))=885
```

--------------------------------

### Find and Extract All Regex Matches

Source: https://documentation.sas.com/doc/en/espcdc/v_071/engelref/n1o0omqusgei9kn1gqnvx7e1rrf9.htm

This example demonstrates how to find all occurrences of a pattern (uppercase letters) in a string using `regex.compile`, `regex.findfirst`, and `regex.findnext`. It extracts each matched substring and records its starting position using `MATCHSTART`. Use `pushrow()` for clarity in examples; remove it and `seteof(true)` if passing data to the node.

```regex
// Define some string variables
string MyString
string MySubString
integer StartLocation

// Set one to some sample input
MyString = "00AA111BBB2222CCCC"
// Will hold the starting location of matched patterns
StartLocation = 0

//You must define a regex object
regex r
// Then compile your regular expression
// This one will match any single uppercase letter
r.compile("[A-Z]+")
// Find the first pattern match
if r.findfirst(MyString)
   begin
   // Pull the pattern from MyString and place it into MySubString
   MySubString = mid(MyString, r.matchstart(),r.matchlength())
   // Use pushrow to create new rows - this is purely for the sake of
   // clarity in the example
   pushrow()
   // Create a while loop that continues to look for matches
   while r.findnext(MyString)
      begin
      // Pull the pattern from MyString and place it into MySubString
      again
      MySubString = mid( MyString, r.matchstart(),r.matchlength())
      // Set StartLocation to the starting point of each pattern found
      StartLocation = r.matchstart()
      // Just for display again
      pushrow()
      end
   end
// Terminate the expression node processing
seteof(true)
// Prevent the last pushrow() from showing up twice
return false
```

--------------------------------

### Mount Volume and Apply License with Docker

Source: https://documentation.sas.com/doc/en/espcdc/v_071/dplyedge0phy0lax/n19xzrpw32y2qrn1w9057vdo946b.htm

Use this command to start the container, mounting a volume for the license file and applying the product license. Ensure the license file is named 'license.txt' and placed in the specified host directory.

```bash
docker run -v _path-to-license-on-host_:/data _registry-name_/_host-name_:_port_/sas-espedge-base:_tag_
```

--------------------------------

### Specify Configuration Section for Adapter

Source: https://documentation.sas.com/doc/en/espcdc/v_071/espca/p0is3bfhfaon4nn1wopxmcbs61nx.htm

This example shows how to use the `configfilesection` parameter to specify a section in a configuration file for the file and socket adapter to use.

```bash
dfesp_fs_adapter -C configfilesection=[adaptercsvAdapterSubConfigfile_input]
```

--------------------------------

### Configure Multiple GPUs for Processing

Source: https://documentation.sas.com/doc/en/espcdc/v_071/espan/n1gk6djwzlve2on1iesy99su5rsi.htm

When using multiple GPUs, specify the number of devices with NDEVICES and then list the specific device IDs starting from 0. This example uses two GPUs with IDs 3 and 5.

```text
i,n,4,USEGPUESP,1
i,n,5,NDEVICES,2
i,n,6,DEVICE0,3
1,n,7,DEVICE1,5
1,n,8,"action","load"
...
```

--------------------------------

### Get File Creation and Modification Dates with Existence Check

Source: https://documentation.sas.com/doc/en/espcdc/v_071/engelref/n00ws3mskqfuzin13wyachp2ngf9.htm

This example demonstrates checking file existence and retrieving both creation and modification dates using FILEEXISTS() and FILEDATE(). It includes a seteof() call, which might be used for cleanup or resetting stream positions.

```EEL
// Expression 
boolean file_test 
date created 
date modified  

file_test=fileexists("C:\filename.txt") 
created=filedate("C:\filename.txt", false) 
modified=filedate("C:\filename.txt", true) 

seteof()
```

--------------------------------

### WRITELINE Function Usage

Source: https://documentation.sas.com/doc/en/espcdc/v_071/espcreatewindows/p0mdw37mzm6x32n1ed5zihe6y9e1.htm

Demonstrates how to use the WRITELINE function to write data to a file. The example shows opening a file in append mode, writing initial text, then using seekbegin(0) to reposition to the start and writing again, illustrating that append mode with seekbegin(0) writes at the beginning.

```APIDOC
## WRITELINE Function

### Description
Writes a line to a file.

### Syntax
fileobject.WRITELINE(<string>)

### Required Argument
#### string
A string specifying the information that needs to be written to the file. This parameter can be specified as a fixed string, field name, or expression.

### Details
The WRITELINE() method writes the string at the current position in the file. This method overwrites data that exists at the current position in the file. If the current position in the file plus the length of the string is larger than the current file size, then the file size is increased.
The file needs to be opened in Write or Append mode for this method to work.

### Example
```
file f

f.open("C:\\filename.txt", "a")
f.writeline("This text will be appended to the file")

f.seekbegin(0)
f.writeline("Using seekbegin(0) and Append will
still cause the info to be written at the start of the file")

f.close()
```

### Returned data type
Integer (1= success, 0 = error)
```

--------------------------------

### Sample Docker Build Output

Source: https://documentation.sas.com/doc/en/espcdc/v_071/espex/n1ilhe30vp1z2zn1chjlzow4tajg.htm

This is sample output from the `docker build` command, showing the steps of building a new image and tagging it.

```text
Sending build context to Docker daemon  956.4kB
Step 1/2 : FROM cr.sas.com/viya-4-x64_oci_linux_2-docker/sas-event-stream-processing-server-app:10.717.27-20210913.1631529109396
---> afba6076e9d1
Step 2/2 : COPY --chown=sas:root libesp_custom_plugin.so /opt/sas/viya/home/SASEventStreamProcessingEngine/lib/plugins
---> 9728a07f06ea
Successfully built 9728a07f06ea
Successfully tagged sas-event-stream-processing-server-app:10.717.27-20210913.1631529109396-custom

```

--------------------------------

### Using VAREVAL to Access Dynamic Field Values

Source: https://documentation.sas.com/doc/en/espcdc/v_071/engelref/n1a6vti3kxdbqyn11mxrkr0hvymh.htm

This example demonstrates how to use the VAREVAL function to iterate through and retrieve values from a series of fields named 'field_1' through 'field_5'. It constructs the field name dynamically and then uses VAREVAL to get its value. Use this function cautiously to avoid performance issues.

```SAS
// Declare the string values for the function
string field_number
string field_value
// Declare a hidden integer as a counter
hidden integer n
// Loop trough all 5 variables in an input data source
for n=1 to 5
// Output the value in each of the fields field_1 through field_5
begin
     field_number='field_' & n
     field_value=vareval(field_number)
     pushrow()
end
// Return false to prevent the last row from showing up twice
return false
```

--------------------------------

### Using MATCHLENGTH to Get Pattern Match Length

Source: https://documentation.sas.com/doc/en/espcdc/v_071/espcreatewindows/n0hdcidjirukq2n17a4001rzcefk.htm

This example demonstrates how to use MATCHLENGTH to find the length of a pattern match. It compiles a regular expression, searches for the first match in a string, and then uses MATCHLENGTH to set a variable to the length of the matched substring. Ensure a regex object is defined and compiled before use.

```SAS
// Define some variables
integer i
string MyString

//Supply some values for the variables
i = 0
MyString = "DataFlux"
// Uncomment the line below to see the value of variable i change
//MyString = "Data_Management_Studio"

//You must define a regex object
regex r
//Then compile your regular expression.
// This expression will match as many "word" characters as it can
// (alphanumerics and undescore)
r.compile("\w*")
// If a pattern match is found then set i to show the length of
// the captured substring
if r.findfirst(MyString) then i = r.matchlength()
// Terminate the expression node processing
seteof()
```

--------------------------------

### publisherStart

Source: https://documentation.sas.com/doc/en/espcdc/v_071/esppsapi/p0tyxhiahf6sa4n1s53ynvzpbl14.htm

Starts a publisher client. Requires server URL, user-defined callbacks, and a context object.

```APIDOC
## publisherStart

### Description
Start a publisher.

### Method Signature
`dfESPclient publisherStart(String _serverURL_, clientCallbacks _userCallbacks_, Object _ctx_)

### Parameters
*   **_serverURL_** (String) - The URL of the server to connect to.
*   **_userCallbacks_** (clientCallbacks) - Callbacks for the user.
*   **_ctx_** (Object) - Context object for the client.
```

--------------------------------

### Publish Tool Configuration Example

Source: https://documentation.sas.com/doc/en/espcdc/v_071/espcreatewindows/p0inznj9984htln1ut77kr3nbu7m.htm

This is an example of a Publish tool configuration. It specifies the input window and optionally includes data content, a URL for data, and a block size for event publishing.

```xml
<tool type="publish" name="..."><title>...</title><description>...</description><config><input window="..."><field>...</field><data>...</data><url>...</url><blocksize>...</blocksize></input></config></tool>
```

```xml
<tool type="publish" name="">1
    <title>...</title>
    <description>...</description>
    <config>
      <input window="">2
        <field>...</field>
        <data>...</data>3
        <url>...</url>4
        <blocksize>...</blocksize>5
      </input>
    </config>
  </tool>
```

--------------------------------

### Generate GUIDs using esp_getGuids

Source: https://documentation.sas.com/doc/en/espcdc/v_071/espcreatewindows/p1rmz15f2xqsqcn1l4r4paxujh6l.htm

Use the esp_getGuids function to retrieve a specified number of GUIDs from the ESP server. The GUIDs are returned in an indexed array. The default number of GUIDs to generate is 1.

```lua
local   guids = esp_getGuids(5)

for index,guid in ipairs(guids) do
    -- Generate event with GUID
end
```

--------------------------------

### Install debsums Utility

Source: https://documentation.sas.com/doc/en/espcdc/v_071/dplyedge0phy0lax/n010f299xbp2len191oeswhyxkdh.htm

Installs the debsums package, which is required for validating MD5 checksums of installed Debian packages.

```bash
sudo apt-get install debsums
```

--------------------------------

### Sample Docker Build Output

Source: https://documentation.sas.com/doc/en/espcdc/v_071/espxmllayer/p1ioa2w4913ottn1bins7awosqmy.htm

This is sample output from the `docker build` command, showing the steps involved in creating the new image.

```text
Sending build context to Docker daemon  956.4kB
Step 1/2 : FROM cr.sas.com/viya-4-x64_oci_linux_2-docker/<deployed_image>:10.717.27-20210914.1631529109396
---> afba6076e9d2
Step 2/2 : COPY --chown=sas:root libesp_custom_plugin.so /opt/sas/viya/home/SASEventStreamProcessingEngine/lib/plugins
---> 9728a07f06eb
Successfully built 9728a07f06eb
Successfully tagged <deployed-image>:10.717.27-20210914.1631529109396-custom
```

--------------------------------

### Override Configuration File Settings with Command-Line Parameters

Source: https://documentation.sas.com/doc/en/espcdc/v_071/espca/p0is3bfhfaon4nn1wopxmcbs61nx.htm

This example demonstrates how the file and socket adapter uses a specified configuration file section, but key-value pairs provided on the command line take precedence.

```bash
dfesp_fs_adapter -C configfileinput=[adaptercsvAdapterSubConfigfile_input],
fsname=TEST_OUTPUT/output/input.out,fstype=csv,dateformat=”%Y-%m-%d %H:%M:%S”
```

--------------------------------

### GUID Function

Source: https://documentation.sas.com/doc/en/espcdc/v_071/espcreatewindows/p1b9sd4ykolfkwn1orjfocca3xs3.htm

Generates and returns a globally unique identifier (GUID).

```esp-cdc
guid()=46ca7b9e-b11d-41be-a3eb-be8bc8553aed
guid()=319cd2a6-1b30-4c1b-8ac7-55e9465ea066
```

--------------------------------

### esp_getGuids Function

Source: https://documentation.sas.com/doc/en/espcdc/v_071/espcreatewindows/p1rmz15f2xqsqcn1l4r4paxujh6l.htm

The esp_getGuids function retrieves a specified number of GUIDs from the ESP server. The GUIDs are returned to Lua in an indexed array. The default value for the number of GUIDs to retrieve is 1.

```APIDOC
## esp_getGuids Function

### Description
Retrieves a specified number of GUIDs from the ESP server. The GUIDs are returned to Lua in an indexed array.

### Parameters
#### Path Parameters
- **_numguids_** (integer) - Optional - The number of GUIDs to retrieve. Defaults to 1.

### Request Example
```lua
local guids = esp_getGuids(5)
for index, guid in ipairs(guids) do
    -- Generate event with GUID
end
```

### Response
#### Success Response (200)
- **guids** (array of strings) - An indexed array containing the generated GUIDs.
```

--------------------------------

### Initialize Publish/Subscribe for a Project

Source: https://documentation.sas.com/doc/en/espcdc/v_071/esppsapi/p0vgqd0q23hgv5n14gs0qvu32yah.htm

Call `project->setPubSub()` before `engine->startProjects()` to initialize publish/subscribe capabilities for a project. `ps_AUTO` enables clients to subscribe to all window output event streams.

```cpp
project->setPubSub(dfESPproject::ps_AUTO);
engine->startProjects();
```

--------------------------------

### Start ESP Standalone Environment

Source: https://documentation.sas.com/doc/en/espcdc/v_071/espex/n1h32i4ez0kadvn1rwbuiaclx2q4.htm

Execute this command to start your ESP standalone environment by scaling up deployed components to their original state. This command assumes the environment was previously stopped using `stop-all`.

```bash
sas-esp-esi.sh workflow --name=start-all run
```

--------------------------------

### Configure Project Selection and Stopping

Source: https://documentation.sas.com/doc/en/espcdc/v_071/esmjt/p1rsxtby0k5cvdn10v8ch3p2chlb.htm

This configuration demonstrates how to prompt the user to select an ESP server and then a running project on that server. It then uses a stop-project instruction to stop the selected project. Ensure the running-project-selector is nested within a server-selector.

```xml
<parameters>
   <server-selector id="serverSelector" 
    localization-id="serverLabel" 
    required="true" 
      <running-project-selector 
       id="runningProjectSelector" 
       localization-id="runningProjectLabel" 
       required="true" />
   </server-selector>
</parameters>
<server-filters>
   <server-filter id="serverFilter"
    filter-expression="(objectId='{serverSelector.id}')" />
<server-filters/>
<instructions>
   <stop-project id="stopProjectInstruction" 
    localization-id="stopProjectLabel" 
    server-filter="serverFilter" 
    running-project-name="{runningProjectSelector.friendlyName}" />
</instructions>
```

--------------------------------

### Example Apache Avro Schema

Source: https://documentation.sas.com/doc/en/espcdc/v_071/esppsapi/n1rwrckgk97eecn1ntdvuus1kw7v.htm

This is an example of an Apache Avro schema for an enhanced event.

```json
{
  "type": "record",
  "name": "EnhancedEvent",
  "namespace": "com.sas.mkt.kafka.domain.events",
  "fields": [
    { 
      "name": "impression", 
      "type": [
        "null",
        {
          "type": "record",
          "name": "Impression",
          "namespace": "com.sas.mkt.kafka.domain.common",
          "fields": [
            { "name": "controlGroup", "type": {"type": "string", "avro.java.string": "String"} },
            { "name": "creativeId", "type": {"type": "string", "avro.java.string": "String"} },
            ...
          ]
        }
      ]
    }
  ]
}
```

--------------------------------

### Example SQL Select Result

Source: https://documentation.sas.com/doc/en/espcdc/v_071/espcreatewindows/p1tmfn8k9g7bmrn1jm84ock9795x.htm

This is an example of the expected return structure from `esp.sqlSelect` when a query is successful.

```json
{
    'code': True,
    'data': (
        {'broker': 101667.0, 'buyer': 1012223.0, 'buysellflg': 0.0, 'currency': 87236.0, 'id': 2669875.0, 'msecs': 418.0, 'price': 4.15, 'quant': 2063400.0, 'seller': 8382739.0, 'symbol': 'C', 'udate': 1280928609000000.0, 'venue': 55555.0},
        {'broker': 101667.0, 'buyer': 0.0, 'buysellflg': 1.0, 'currency': 87236.0, 'id': 18564845.0, 'msecs': 724.0, 'price': 21.7, 'quant': 1286200.0, 'seller': 8123541.0, 'symbol': 'XEL', 'udate': 1280928728000000.0, 'venue': 55555.0}
    )
}
```

--------------------------------

### Subscribe, Publish, and Unsubscribe with esp_subscribe and esp_publish

Source: https://documentation.sas.com/doc/en/espcdc/v_071/espcreatewindows/p1rmz15f2xqsqcn1l4r4paxujh6l.htm

An example demonstrating how to subscribe to one window, process events, publish them to another window, and then unsubscribe.

```lua
local subscriber = nil

function init()
    subscriber = esp_subscribe({window="p/cq/alert",handler="handler"})
end

function handler(events)
    esp_publish({window="cq2/storeAlerts",events=events,opts={logs=true}})
end

function done()
    esp_unsubscribe(subscriber)
end
```

--------------------------------

### Kafka Adapter Command Line Usage

Source: https://documentation.sas.com/doc/en/espcdc/v_071/espca/p0sbfix2ql9xpln1l1x4t9017aql.htm

Example of how to invoke the Kafka adapter from the command line with configuration parameters.

```bash
dfesp_kafka_adapter -C _key1_ =_val1_ ,_key2_ =_val2_ ,_key3_ =_val3_ ,...
```

--------------------------------

### JSON Array Example

Source: https://documentation.sas.com/doc/en/espcdc/v_071/espcreatewindows/p0xnrhwj458ygqn0zg6kckuw4ze4.htm

This is an example of a JSON array structure that might be found in an event field.

```json
[
    {
        "device-id":1000,
        "sensor-id":10,
        "value":1.54
    },
    {
        "device-id":1000,
        "sensor-id":20,
        "value":9.23
    },
    {
        "device-id":2000,
        "sensor-id":10,
        "value":3.24
    }
]
```

--------------------------------

### Set Up Publisher

Source: https://documentation.sas.com/doc/en/espcdc/v_071/espws/p18hscckw54rqqn1813by2r88jqb.htm

Encodes and sends a request to set up a publisher for a specific window. The 'schema' parameter should be set to 'true' to include schema information.

```javascript
request = {
        "publisher":{
            "id":"publish",
            "action":"set",
            "window":"myproject/cq/clicks",
            "schema":"true"
        }
    };
    ws.send(api.encode(request))
```

--------------------------------

### Example Input Event

Source: https://documentation.sas.com/doc/en/espcdc/v_071/espcreatewindows/p0uxkz3ve828tmn1ijbd0tu5v23d.htm

An example of a wide event streamed into a Source window, which is then processed by a Transpose window.

```text
I N 1 turboprop #2 1.2 21:00.0 2.2 21:00.0 -1.2 21:00.0 -1.2 21:00.0 20 20
```

--------------------------------

### Basic C++ or Java Adapter Command-Line Syntax

Source: https://documentation.sas.com/doc/en/espcdc/v_071/espca/p0is3bfhfaon4nn1wopxmcbs61nx.htm

Use this syntax to run C++ or Java adapters. Specify comma-separated key-value pairs to control adapter behavior, including a `configfilesection` to reference a section in a configuration file.

```bash
dfesp__adapter_ _adapter -C _key1_ =_val1_ ,_key2_ =_val2_ , ...
```

--------------------------------

### Project Configuration with Procedural Window

Source: https://documentation.sas.com/doc/en/espcdc/v_071/espcreatewindows/n06blye2qhhz55n1fityyts39qsk.htm

This XML configuration sets up a project with a source window reading from 'input.csv' and a procedural window that uses a C++ plugin to count opcodes from the source window.

```xml
<project name='project' pubsub='auto' threads='1'>
   <contqueries>
    <contquery name='contQuery' trace='proceduralWindow'>
     <windows>
      <window-source name='sourceWindow'>
       <schema>
        <fields>
         <field name='ID' type='int32' key='true'/>
         <field name='symbol' type='string'/>
         <field name='quantity' type='int32'/>
         <field name='price' type='double'/>
        </fields>
       </schema>
       <connectors>
        <connector class='fs' name='publisher'>
         <properties>
          <property name='type'>pub</property>
          <property name='fstype'>csv</property>
          <property name='fsname'>input.csv</property>
          <property name='transactional'>true</property>
          <property name='blocksize'>1</property>
         </properties>
        </connector>
       </connectors>
      </window-source>
      <window-procedural name='proceduralWindow'>
       <schema>
        <fields>
         <field name='ID' type='int32' key='true'/>
         <field name='insertCount' type='int32'/>
         <field name='updateCount' type='int32'/>
         <field name='deleteCount' type='int32'/>
        </fields>
       </schema>
       <cxx-plugin-context name='plugin' function='get_derived_context'/>
       <cxx-plugin source='sourceWindow' name='plugin' function='countOpcodes'/>
      </window-procedural>
     </windows>
     <edges>
      <edge source='sourceWindow' target='proceduralWindow'/>
     </edges>
    </contquery>
   </contqueries>
  </project>
```

--------------------------------

### XML Sensor Data Example

Source: https://documentation.sas.com/doc/en/espcdc/v_071/espcreatewindows/n1pz8fjb7urz3ln1vuovreq4yng2.htm

This is an example of sensor data in XML format that might arrive in a field.

```xml
<data>
    <sensorinfo>
        <device_id>1000</device_id>
        <sensor_id>10</sensor_id>
        <value>1.54</value>
    </sensorinfo>
    <sensorinfo>
        <device_id>1000</device_id>
        <sensor_id>20</sensor_id>
        <value>9.23</value>
    </sensorinfo>
    <sensorinfo>
        <device_id>2000</device_id>
        <sensor_id>10</sensor_id>
        <value>3.24</value>
    </sensorinfo>
</data>
```

--------------------------------

### Enable ESP Server Start/Stop

Source: https://documentation.sas.com/doc/en/espcdc/v_071/espex/n19tbdmek5u0rdn1f31lktl95r27.htm

Use this command to enable an ESP server to opt in to starting and stopping. Ensure the correct namespace and server name are used.

```bash
kubectl label \
  espservers.iot.sas.com/$SAS_ESP_SERVER_NAME \
  esp.iot.sas.com/server-pause-enable- \
  --namespace=$K8S_NAMESPACE \
  --overwrite
```
Tokens:
Raw