Content Insight Agent

https://www.volcengine.com/docs/83600/
Admin
Content Insight Agent is a platform for monitoring and analyzing content data with real-time task...
Tokens:1,454
Snippets:4
Trust Score:10
Update:10 months ago
Show doc for...
Tokens:
Raw
### Get Volcengine Access Token

Source: https://www.volcengine.com/docs/83600/1200118

Retrieves an access token from the Volcengine authentication server. It constructs an HTTP GET request with necessary headers like business name and secret, then sends the request and unmarshals the JSON response to extract the access token and its expiry time. The token and expiry are stored globally.

```go
// GetAccessToken Generates access token method
//
//  @param baseURL string
//  @param bizName string
//  @return err error
//  @update 2023-08-07 03:23:43
func GetAccessToken() (err error) {
    c, _ := client.NewClient(client.WithTLSConfig(&tls.Config{
        InsecureSkipVerify: true,
    }))
    req := &protocol.Request{}
    req.SetMethod(consts.MethodGet)
    req.SetHeaders(
        map[string]string{
            "X-Real-Ip":            "180.184.86.50",
            "X-Insight-Biz-Name":   volcBizName,
            "X-Insight-Biz-Secret": volcSecret,
        },
    )
    req.SetRequestURI(insightBaseURL + oauthTokenPath)
    rsp := &protocol.Response{}
    if err := c.Do(context.Background(), req, rsp); err != nil {
        logger.Error(err.Error())
    }
    tokenResp := &AccessTokenResp{}
    json.Unmarshal(rsp.BodyBytes(), &tokenResp)
    AccessToken.token = tokenResp.Data.AccessToken
    AccessToken.expireAt = time.Now().Local().Add(time.Duration(tokenResp.Data.Expire) * time.Second)
    return
}
```

--------------------------------

### Subscribe to SSE Stream and Decrypt Data

Source: https://www.volcengine.com/docs/83600/1200118

Handles receiving Server-Sent Events (SSE) from a subscription stream. It builds an HTTP GET request, reads the response body as a stream, and processes each line. Data fields are unmarshalled from JSON, decrypted using CBC mode with AES, and the decrypted message is then available for further processing. Includes error handling for network requests and JSON unmarshalling.

```go
func (ctx *subMsgContext) subMsg() {
    defer ctx.recoverSubMsg()
    ctx.RefreshToken(false)
    var (
        req = &protocol.Request{}
        rsp = &protocol.Response{}
        err error
    )

    // building http request
    req.SetMethod(consts.MethodGet)
    req.SetRequestURI(insightBaseURL + streamReqPath)
    queryStr := "collector=" + ctx.collectors
    if ctx.startOffset != 0 {
        queryStr += "&start_offset=" + strconv.FormatInt(ctx.startOffset, 10)
    }
    req.SetQueryString(queryStr)
    req.SetHeaders(
        map[string]string{
            "X-Real-Ip":              "180.184.86.50",
            "X-Insight-Biz-Name":     volcBizName,
            "X-Insight-Access-Token": ctx.token,
        },
    )

    c, err := client.NewClient(
        client.WithResponseBodyStream(true),
        client.WithTLSConfig(&tls.Config{
            InsecureSkipVerify: true,
        }),
    )
    if err != nil {
        logger.Panic(err)
    }
    err = c.Do(ctx, req, rsp)
    if err != nil {
        logger.Panic(err)
    }

    // Create a new scanner to read the SSE data as a stream
    const maxTokenSize = 1024 * 1024 // 1 MB
    scanner := bufio.NewScanner(rsp.BodyStream())
    scanner.Buffer(make([]byte, maxTokenSize), maxTokenSize)
    // Set the scanner to split on newlines
    scanner.Split(bufio.ScanLines)

    key := []byte(volcAesKey)
    // Loop to read SSE events from the scanner
    for scanner.Scan() {
        line := scanner.Text()
        // Check if the line is a comment or a data line
        if len(line) == 0 || line[0] == ':' {
            continue
        }

        // Split the line into event and data fields
        for _, field := range splitFields(line) {
            if field != "" {
                var bizOutItem *StreamMessage
                if field == "unauthorized" {
                    logger.Panic(field)
                }
                err = json.Unmarshal([]byte(field), &bizOutItem)
                if err != nil {
                    logger.Warn("json unmarshal %+v", err)
                }
                iv := bizOutItem.Nonce
                decryptMsg, err := CBCDecrypter(bizOutItem.EncryptItem, key, iv)
                if err != nil {
                    logger.Warn("decrypt message err %+v, msg %v", err, decryptMsg)
                }
                // logger.Infof("recv: %s %s", ctx.collectors, decryptMsg[:10])
                // TODO: Here's where the post decrypted, capture the post here.
            }
        }
    }

    // Check if there was an error reading from the scanner
    if err := scanner.Err(); err != nil {
        logger.Panic(err)
    }
}
```

--------------------------------

### CBC Decrypt Data with AES

Source: https://www.volcengine.com/docs/83600/1200118

Performs CBC decryption on given ciphertext using AES. It initializes a new AES cipher with the provided key, creates a CBC decrypter, and then decrypts the data. Finally, it removes PKCS7 padding from the decrypted result. Errors during cipher initialization or decryption are handled.

```go
// CBCDecrypter CBC 解密 [copied from utils]
//
//  @param encrypter []byte 待解密的密文
//  @param key []byte 秘钥
//  @param iv []byte
//  @return result []byte
//  @return err error
//  @update 2023-08-14 07:35:00
func CBCDecrypter(encrypter []byte, key []byte, iv []byte) (result []byte, err error) {
    block, err := aes.NewCipher(key)
    if err != nil {
        return result, errors.Wrap(err, "aes decipher err")
    }
    blockMode := cipher.NewCBCDecrypter(block, iv)
    result = make([]byte, len(encrypter))
    blockMode.CryptBlocks(result, encrypter)
    // 去除填充
    result = UnPKCS7Padding(result)
    return result, err
}
```

--------------------------------

### Remove PKCS7 Padding

Source: https://www.volcengine.com/docs/83600/1200118

Removes PKCS7 padding from a byte slice. This function is typically used after decryption when the data has been padded according to the PKCS7 standard. It returns the original data without the padding bytes.

```go
// UnPKCS7Padding 去除 PKCS7Padding 填充的数据 [copied from utils]
//
//  @param text []byte 待去除填充数据的原文
//  @return []byte
//  @update 2023-08-14 07:35:33

```
Tokens:
Raw