# Certificate Transparency Go

This repository provides a comprehensive Go implementation of Certificate Transparency (CT) as defined in RFC 6962. Certificate Transparency is a critical security technology that creates an open framework for monitoring and auditing SSL/TLS certificates in near real-time. The system enables domain owners, certificate authorities, and security researchers to detect mistakenly or maliciously issued certificates, thereby strengthening the overall web security ecosystem.

The library includes complete implementations of CT log clients for querying logs via HTTP and DNS, tools for scanning entire logs, cryptographic verification utilities for SCTs (Signed Certificate Timestamps) and STHs (Signed Tree Heads), policy enforcement engines for Chrome and Apple CT requirements, and a production-ready CT log server implementation built on the Trillian transparency log infrastructure. It also provides specialized X.509 and ASN.1 parsing libraries designed to handle malformed certificates that exist in the wild, making it suitable for ecosystem-wide monitoring and analysis.

## Core Data Types and Structures

### SignedCertificateTimestamp (SCT)

An SCT is a cryptographic promise from a CT log to incorporate a certificate into the log within a maximum merge delay period.

```go
package main

import (
    "encoding/base64"
    "fmt"
    "log"

    ct "github.com/google/certificate-transparency-go"
    "github.com/google/certificate-transparency-go/tls"
)

func main() {
    // Parse an SCT from base64-encoded data received from a CT log
    sctData := "AFUBo0BaKEYFd8PnJ5XvQJXKReDQXMZxHwfTdLJMOWQ="
    rawSCT, err := base64.StdEncoding.DecodeString(sctData)
    if err != nil {
        log.Fatal(err)
    }

    var sct ct.SignedCertificateTimestamp
    rest, err := tls.Unmarshal(rawSCT, &sct)
    if err != nil {
        log.Fatalf("Failed to unmarshal SCT: %v", err)
    }
    if len(rest) > 0 {
        log.Printf("Warning: %d trailing bytes after SCT", len(rest))
    }

    fmt.Printf("SCT Version: %v\n", sct.SCTVersion)
    fmt.Printf("Log ID: %x\n", sct.LogID.KeyID[:])
    fmt.Printf("Timestamp: %d\n", sct.Timestamp)
    fmt.Printf("Extensions: %x\n", sct.Extensions)
    fmt.Printf("Signature Algorithm: %v\n", sct.Signature)
}
```

### SignedTreeHead (STH)

An STH represents a cryptographically signed snapshot of the log's Merkle tree at a specific size and time.

```go
package main

import (
    "context"
    "crypto/sha256"
    "fmt"
    "log"
    "net/http"
    "time"

    ct "github.com/google/certificate-transparency-go"
    "github.com/google/certificate-transparency-go/client"
    "github.com/google/certificate-transparency-go/jsonclient"
)

func main() {
    // Create a client for Google's Argon2024 log
    logURL := "https://ct.googleapis.com/logs/us1/argon2024/"
    httpClient := &http.Client{Timeout: 30 * time.Second}

    logClient, err := client.New(logURL, httpClient, jsonclient.Options{})
    if err != nil {
        log.Fatalf("Failed to create log client: %v", err)
    }

    ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
    defer cancel()

    // Retrieve the current signed tree head
    sth, err := logClient.GetSTH(ctx)
    if err != nil {
        log.Fatalf("Failed to get STH: %v", err)
    }

    fmt.Printf("Tree Size: %d\n", sth.TreeSize)
    fmt.Printf("Timestamp: %v\n", ct.TimestampToTime(sth.Timestamp))
    fmt.Printf("Root Hash: %x\n", sth.SHA256RootHash)
    fmt.Printf("Signature Algorithm: %v/%v\n",
        sth.TreeHeadSignature.Algorithm.Hash,
        sth.TreeHeadSignature.Algorithm.Signature)
}
```

### MerkleTreeLeaf Construction

Create Merkle tree leaf entries for X.509 certificates and precertificates.

```go
package main

import (
    "crypto/x509"
    "fmt"
    "log"
    "time"

    ct "github.com/google/certificate-transparency-go"
    x509ct "github.com/google/certificate-transparency-go/x509"
)

func main() {
    // Parse a certificate (in production, load from PEM file)
    certPEM := []byte(`-----BEGIN CERTIFICATE-----
MIIBkTCB+wIJAKHHCgVZU7DGMA0GCSqGSIb3DQEBCwUAMBExDzANBgNVBAMMBnRl
c3RjYTAeFw0yNDAxMDEwMDAwMDBaFw0yNTAxMDEwMDAwMDBaMBExDzANBgNVBAMM
BnRlc3RjYTCBnzANBgkqhkiG9w0BAQEFAAOBjQAwgYkCgYEAyW1hx3u7bxF2+KlI
-----END CERTIFICATE-----`)

    cert, err := x509ct.ParseCertificate(certPEM)
    if err != nil {
        log.Fatalf("Failed to parse certificate: %v", err)
    }

    // Create a Merkle tree leaf for an X.509 certificate
    timestamp := uint64(time.Now().UnixNano() / 1000000) // milliseconds since epoch
    leaf := ct.CreateX509MerkleTreeLeaf(ct.ASN1Cert{Data: cert.Raw}, timestamp)

    fmt.Printf("Leaf Version: %v\n", leaf.Version)
    fmt.Printf("Leaf Type: %v\n", leaf.LeafType)
    fmt.Printf("Entry Type: %v\n", leaf.TimestampedEntry.EntryType)
    fmt.Printf("Timestamp: %d\n", leaf.TimestampedEntry.Timestamp)

    // Calculate the leaf hash
    leafHash, err := ct.LeafHashForLeaf(leaf)
    if err != nil {
        log.Fatalf("Failed to calculate leaf hash: %v", err)
    }
    fmt.Printf("Leaf Hash: %x\n", leafHash)
}
```

## CT Log Client Operations

### Retrieving and Verifying Signed Tree Heads

Query a CT log for its current tree head and verify the signature.

```go
package main

import (
    "context"
    "crypto"
    "encoding/base64"
    "fmt"
    "log"
    "net/http"
    "time"

    ct "github.com/google/certificate-transparency-go"
    "github.com/google/certificate-transparency-go/client"
    "github.com/google/certificate-transparency-go/jsonclient"
)

func main() {
    // Log's public key (base64-encoded DER)
    pubKeyB64 := "MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAE1HVVnZ4..." // truncated for brevity
    pubKeyDER, err := base64.StdEncoding.DecodeString(pubKeyB64)
    if err != nil {
        log.Fatalf("Failed to decode public key: %v", err)
    }

    // Create a client with signature verification enabled
    logURL := "https://ct.googleapis.com/logs/us1/argon2024/"
    httpClient := &http.Client{Timeout: 30 * time.Second}

    logClient, err := client.New(logURL, httpClient, jsonclient.Options{
        PublicKeyDER: pubKeyDER,
    })
    if err != nil {
        log.Fatalf("Failed to create client: %v", err)
    }

    ctx := context.Background()

    // Get and automatically verify STH
    sth, err := logClient.GetSTH(ctx)
    if err != nil {
        log.Fatalf("Failed to get/verify STH: %v", err)
    }

    fmt.Printf("✓ STH signature verified\n")
    fmt.Printf("Tree contains %d certificates\n", sth.TreeSize)
    fmt.Printf("Last updated: %v\n", ct.TimestampToTime(sth.Timestamp))
}
```

### Submitting Certificates and Precertificates

Submit certificate chains to a CT log and receive SCTs.

```go
package main

import (
    "context"
    "crypto/x509"
    "encoding/pem"
    "fmt"
    "log"
    "net/http"
    "os"
    "time"

    ct "github.com/google/certificate-transparency-go"
    "github.com/google/certificate-transparency-go/client"
    "github.com/google/certificate-transparency-go/jsonclient"
)

func main() {
    // Load certificate chain from files
    certPEM, err := os.ReadFile("leaf-cert.pem")
    if err != nil {
        log.Fatalf("Failed to read cert: %v", err)
    }

    issuerPEM, err := os.ReadFile("issuer-cert.pem")
    if err != nil {
        log.Fatalf("Failed to read issuer: %v", err)
    }

    // Parse PEM blocks
    certBlock, _ := pem.Decode(certPEM)
    issuerBlock, _ := pem.Decode(issuerPEM)

    // Build chain for submission
    chain := []ct.ASN1Cert{
        {Data: certBlock.Bytes},
        {Data: issuerBlock.Bytes},
    }

    // Create client
    logURL := "https://ct.googleapis.com/logs/us1/argon2024/"
    httpClient := &http.Client{Timeout: 30 * time.Second}

    logClient, err := client.New(logURL, httpClient, jsonclient.Options{})
    if err != nil {
        log.Fatalf("Failed to create client: %v", err)
    }

    ctx, cancel := context.WithTimeout(context.Background(), 20*time.Second)
    defer cancel()

    // Submit certificate chain
    sct, err := logClient.AddChain(ctx, chain)
    if err != nil {
        log.Fatalf("Failed to add chain: %v", err)
    }

    fmt.Printf("✓ Certificate submitted successfully\n")
    fmt.Printf("SCT Timestamp: %v\n", ct.TimestampToTime(sct.Timestamp))
    fmt.Printf("SCT Log ID: %x\n", sct.LogID.KeyID[:8])

    // For precertificates, use AddPreChain instead:
    // sct, err := logClient.AddPreChain(ctx, chain)
}
```

### Retrieving Log Entries

Fetch and parse entries from a CT log by index range.

```go
package main

import (
    "context"
    "fmt"
    "log"
    "net/http"
    "time"

    ct "github.com/google/certificate-transparency-go"
    "github.com/google/certificate-transparency-go/client"
    "github.com/google/certificate-transparency-go/jsonclient"
    "github.com/google/certificate-transparency-go/x509"
)

func main() {
    logURL := "https://ct.googleapis.com/logs/us1/argon2024/"
    httpClient := &http.Client{Timeout: 30 * time.Second}

    logClient, err := client.New(logURL, httpClient, jsonclient.Options{})
    if err != nil {
        log.Fatalf("Failed to create client: %v", err)
    }

    ctx := context.Background()

    // Get entries 100-104
    entries, err := client.GetEntries(ctx, logClient, 100, 104)
    if err != nil {
        log.Fatalf("Failed to get entries: %v", err)
    }

    fmt.Printf("Retrieved %d entries\n\n", len(entries))

    for i, entry := range entries {
        // Parse the raw entry into a structured log entry
        logEntry, err := ct.LogEntryFromLeaf(entry.Index, &entry.LeafEntry)
        if err != nil {
            log.Printf("Entry %d: parsing error: %v", entry.Index, err)
            continue
        }

        fmt.Printf("Entry %d:\n", entry.Index)

        switch logEntry.Leaf.TimestampedEntry.EntryType {
        case ct.X509LogEntryType:
            if logEntry.X509Cert != nil {
                fmt.Printf("  Type: X.509 Certificate\n")
                fmt.Printf("  Subject: %s\n", logEntry.X509Cert.Subject)
                fmt.Printf("  Issuer: %s\n", logEntry.X509Cert.Issuer)
                fmt.Printf("  NotBefore: %v\n", logEntry.X509Cert.NotBefore)
                fmt.Printf("  NotAfter: %v\n", logEntry.X509Cert.NotAfter)
                fmt.Printf("  DNS Names: %v\n", logEntry.X509Cert.DNSNames)
            }
        case ct.PrecertLogEntryType:
            if logEntry.Precert != nil && logEntry.Precert.TBSCertificate != nil {
                fmt.Printf("  Type: Precertificate\n")
                fmt.Printf("  Subject: %s\n", logEntry.Precert.TBSCertificate.Subject)
                fmt.Printf("  DNS Names: %v\n", logEntry.Precert.TBSCertificate.DNSNames)
            }
        }
        fmt.Println()
    }
}
```

### Verifying Consistency Proofs

Verify that a log has grown append-only by checking consistency proofs between tree heads.

```go
package main

import (
    "context"
    "crypto/sha256"
    "encoding/hex"
    "fmt"
    "log"
    "net/http"
    "time"

    ct "github.com/google/certificate-transparency-go"
    "github.com/google/certificate-transparency-go/client"
    "github.com/google/certificate-transparency-go/jsonclient"
    "github.com/google/trillian/merkle"
    "github.com/google/trillian/merkle/rfc6962"
)

func main() {
    logURL := "https://ct.googleapis.com/logs/us1/argon2024/"
    httpClient := &http.Client{Timeout: 30 * time.Second}

    logClient, err := client.New(logURL, httpClient, jsonclient.Options{})
    if err != nil {
        log.Fatalf("Failed to create client: %v", err)
    }

    ctx := context.Background()

    // Simulate having an old STH and getting a new one
    oldTreeSize := uint64(1000000)
    oldRootHash, _ := hex.DecodeString("a3b2c1d4e5f6...") // Previous root hash

    // Get current STH
    newSTH, err := logClient.GetSTH(ctx)
    if err != nil {
        log.Fatalf("Failed to get STH: %v", err)
    }

    if newSTH.TreeSize < oldTreeSize {
        log.Fatal("Tree has shrunk - this should never happen!")
    }

    // Get consistency proof
    proof, err := logClient.GetSTHConsistency(ctx, oldTreeSize, newSTH.TreeSize)
    if err != nil {
        log.Fatalf("Failed to get consistency proof: %v", err)
    }

    // Verify the consistency proof
    hasher := rfc6962.DefaultHasher
    verifier := merkle.NewLogVerifier(hasher)

    err = verifier.VerifyConsistencyProof(
        int64(oldTreeSize),
        int64(newSTH.TreeSize),
        oldRootHash,
        newSTH.SHA256RootHash[:],
        proof,
    )

    if err != nil {
        log.Fatalf("Consistency proof verification failed: %v", err)
    }

    fmt.Printf("✓ Consistency proof verified\n")
    fmt.Printf("Log grew from %d to %d entries\n", oldTreeSize, newSTH.TreeSize)
    fmt.Printf("New root hash: %x\n", newSTH.SHA256RootHash)
}
```

### Verifying Inclusion Proofs

Verify that a specific certificate is included in the log's Merkle tree.

```go
package main

import (
    "context"
    "crypto/sha256"
    "encoding/base64"
    "fmt"
    "log"
    "net/http"
    "time"

    ct "github.com/google/certificate-transparency-go"
    "github.com/google/certificate-transparency-go/client"
    "github.com/google/certificate-transparency-go/jsonclient"
    "github.com/google/trillian/merkle"
    "github.com/google/trillian/merkle/rfc6962"
)

func main() {
    logURL := "https://ct.googleapis.com/logs/us1/argon2024/"
    httpClient := &http.Client{Timeout: 30 * time.Second}

    logClient, err := client.New(logURL, httpClient, jsonclient.Options{})
    if err != nil {
        log.Fatalf("Failed to create client: %v", err)
    }

    ctx := context.Background()

    // Get current STH for tree size
    sth, err := logClient.GetSTH(ctx)
    if err != nil {
        log.Fatalf("Failed to get STH: %v", err)
    }

    // Leaf hash to verify (obtained from SCT or previous query)
    leafHashB64 := "x5YxB3AwG7WEEPGm3q8p8h3h6qH5D2qGFG3Q+qH5D2o="
    leafHash, err := base64.StdEncoding.DecodeString(leafHashB64)
    if err != nil {
        log.Fatalf("Failed to decode leaf hash: %v", err)
    }

    // Get proof by hash
    proofResp, err := logClient.GetProofByHash(ctx, leafHash, sth.TreeSize)
    if err != nil {
        log.Fatalf("Failed to get proof: %v", err)
    }

    // Verify the inclusion proof
    hasher := rfc6962.DefaultHasher
    verifier := merkle.NewLogVerifier(hasher)

    err = verifier.VerifyInclusionProof(
        proofResp.LeafIndex,
        int64(sth.TreeSize),
        proofResp.AuditPath,
        sth.SHA256RootHash[:],
        leafHash,
    )

    if err != nil {
        log.Fatalf("Inclusion proof verification failed: %v", err)
    }

    fmt.Printf("✓ Inclusion proof verified\n")
    fmt.Printf("Certificate at index %d is in tree of size %d\n",
        proofResp.LeafIndex, sth.TreeSize)
}
```

## Cryptographic Verification

### Verifying SCT Signatures

Verify that an SCT was issued by a specific log and is valid for a given certificate.

```go
package main

import (
    "crypto"
    "encoding/base64"
    "fmt"
    "log"
    "os"

    ct "github.com/google/certificate-transparency-go"
    "github.com/google/certificate-transparency-go/ctutil"
    "github.com/google/certificate-transparency-go/x509"
)

func main() {
    // Load the CT log's public key
    pubKeyPEM := []byte(`-----BEGIN PUBLIC KEY-----
MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAE1HVVnZ4UBPxqLYqPq7J3m8h8k5p6
7Gq9j8h3m9j3k5p67Gq9j8h3m9j3k5p67Gq9j8h3m9j3k5p67Gq9j8h3m9==
-----END PUBLIC KEY-----`)

    pubKey, _, _, err := ct.PublicKeyFromPEM(pubKeyPEM)
    if err != nil {
        log.Fatalf("Failed to parse public key: %v", err)
    }

    // Load certificate
    certPEM, err := os.ReadFile("cert.pem")
    if err != nil {
        log.Fatalf("Failed to read cert: %v", err)
    }

    cert, err := x509.ParseCertificate(certPEM)
    if err != nil {
        log.Fatalf("Failed to parse cert: %v", err)
    }

    // Parse SCT (received from log or extracted from certificate)
    sctB64 := "AFUBo0BaKEYFd8PnJ5XvQJXKReDQXMZxHwfTdLJMOWQ=..."
    var sct ct.SignedCertificateTimestamp
    err = (&sct).FromBase64String(sctB64)
    if err != nil {
        log.Fatalf("Failed to parse SCT: %v", err)
    }

    // Verify SCT for a regular X.509 certificate
    chain := []*x509.Certificate{cert}
    err = ctutil.VerifySCT(pubKey, chain, &sct, false)
    if err != nil {
        log.Fatalf("SCT verification failed: %v", err)
    }

    fmt.Printf("✓ SCT signature verified\n")
    fmt.Printf("Certificate: %s\n", cert.Subject)
    fmt.Printf("SCT Timestamp: %v\n", ct.TimestampToTime(sct.Timestamp))

    // For embedded SCTs in certificates, use embedded=true:
    // err = ctutil.VerifySCT(pubKey, chain, &sct, true)
}
```

### Creating and Verifying Signature Input

Serialize data structures for signature creation and verification.

```go
package main

import (
    "crypto"
    "crypto/ecdsa"
    "crypto/rand"
    "crypto/sha256"
    "encoding/asn1"
    "fmt"
    "log"

    ct "github.com/google/certificate-transparency-go"
    "github.com/google/certificate-transparency-go/tls"
    "github.com/google/certificate-transparency-go/x509"
)

func main() {
    // Example: Verify STH signature manually

    // STH data structure
    sth := ct.SignedTreeHead{
        Version:        ct.V1,
        TreeSize:       1000000,
        Timestamp:      1704067200000, // Jan 1, 2024
        SHA256RootHash: ct.SHA256Hash{0x12, 0x34, /* ... */},
    }

    // Serialize the STH for signature verification
    serialized, err := ct.SerializeSTHSignatureInput(sth)
    if err != nil {
        log.Fatalf("Failed to serialize STH: %v", err)
    }

    fmt.Printf("Serialized STH (%d bytes): %x\n", len(serialized), serialized)

    // The signature would be verified using:
    // verifier := ct.NewSignatureVerifier(publicKey)
    // err = verifier.VerifySTHSignature(sth)

    // Example: Create SCT signature input
    cert := &x509.Certificate{ /* certificate data */ }
    chain := []*x509.Certificate{cert}

    sct := ct.SignedCertificateTimestamp{
        SCTVersion: ct.V1,
        Timestamp:  1704067200000,
        Extensions: ct.CTExtensions{},
    }

    // Create Merkle tree leaf
    leaf, err := ct.MerkleTreeLeafFromChain(chain, ct.X509LogEntryType, sct.Timestamp)
    if err != nil {
        log.Fatalf("Failed to create leaf: %v", err)
    }

    entry := ct.LogEntry{Leaf: *leaf}

    // Serialize for signing
    sctInput, err := ct.SerializeSCTSignatureInput(sct, entry)
    if err != nil {
        log.Fatalf("Failed to serialize SCT input: %v", err)
    }

    fmt.Printf("SCT signature input (%d bytes): %x\n", len(sctInput), sctInput)
}
```

## Scanning and Monitoring Logs

### Scanning a Complete Log

Scan all entries in a CT log with custom matching logic.

```go
package main

import (
    "context"
    "fmt"
    "log"
    "net/http"
    "strings"
    "time"

    ct "github.com/google/certificate-transparency-go"
    "github.com/google/certificate-transparency-go/client"
    "github.com/google/certificate-transparency-go/jsonclient"
    "github.com/google/certificate-transparency-go/scanner"
    "github.com/google/certificate-transparency-go/x509"
)

// Custom matcher to find certificates for a specific domain
type DomainMatcher struct {
    domain       string
    matchCount   int
}

func (m *DomainMatcher) CertificateMatches(cert *x509.Certificate) bool {
    // Check if certificate is for our domain
    for _, dns := range cert.DNSNames {
        if strings.Contains(strings.ToLower(dns), m.domain) {
            return true
        }
    }
    if strings.Contains(strings.ToLower(cert.Subject.CommonName), m.domain) {
        return true
    }
    return false
}

func (m *DomainMatcher) PrecertificateMatches(precert *ct.Precertificate) bool {
    if precert.TBSCertificate == nil {
        return false
    }

    for _, dns := range precert.TBSCertificate.DNSNames {
        if strings.Contains(strings.ToLower(dns), m.domain) {
            return true
        }
    }
    return false
}

func main() {
    logURL := "https://ct.googleapis.com/logs/us1/argon2024/"
    httpClient := &http.Client{Timeout: 30 * time.Second}

    logClient, err := client.New(logURL, httpClient, jsonclient.Options{})
    if err != nil {
        log.Fatalf("Failed to create client: %v", err)
    }

    // Configure scanner
    matcher := &DomainMatcher{domain: "example.com"}

    opts := scanner.ScannerOptions{
        Matcher:    matcher,
        NumWorkers: 4,
        BufferSize: 1000,
        FetcherOptions: scanner.FetcherOptions{
            StartIndex:   0,
            EndIndex:     10000, // Scan first 10k entries
            BatchSize:    100,
            ParallelFetch: 2,
        },
    }

    s := scanner.NewScanner(logClient, opts)

    ctx, cancel := context.WithTimeout(context.Background(), 10*time.Minute)
    defer cancel()

    foundCerts := make([]*ct.RawLogEntry, 0)
    foundPrecerts := make([]*ct.RawLogEntry, 0)

    // Scan the log
    err = s.Scan(ctx,
        func(entry *ct.RawLogEntry) {
            foundCerts = append(foundCerts, entry)
            log.Printf("Found cert at index %d", entry.Index)
        },
        func(entry *ct.RawLogEntry) {
            foundPrecerts = append(foundPrecerts, entry)
            log.Printf("Found precert at index %d", entry.Index)
        },
    )

    if err != nil {
        log.Fatalf("Scan failed: %v", err)
    }

    fmt.Printf("\nScan complete!\n")
    fmt.Printf("Found %d certificates\n", len(foundCerts))
    fmt.Printf("Found %d precertificates\n", len(foundPrecerts))
}
```

### Incremental Log Monitoring

Monitor a log for new entries since the last check.

```go
package main

import (
    "context"
    "database/sql"
    "fmt"
    "log"
    "net/http"
    "time"

    ct "github.com/google/certificate-transparency-go"
    "github.com/google/certificate-transparency-go/client"
    "github.com/google/certificate-transparency-go/jsonclient"
    "github.com/google/certificate-transparency-go/scanner"
    _ "github.com/mattn/go-sqlite3"
)

func main() {
    // Open database to track scan progress
    db, err := sql.Open("sqlite3", "ct_monitor.db")
    if err != nil {
        log.Fatal(err)
    }
    defer db.Close()

    // Create table for tracking
    _, err = db.Exec(`
        CREATE TABLE IF NOT EXISTS log_progress (
            log_url TEXT PRIMARY KEY,
            last_index INTEGER,
            last_update TIMESTAMP
        )
    `)
    if err != nil {
        log.Fatal(err)
    }

    logURL := "https://ct.googleapis.com/logs/us1/argon2024/"
    httpClient := &http.Client{Timeout: 30 * time.Second}

    logClient, err := client.New(logURL, httpClient, jsonclient.Options{})
    if err != nil {
        log.Fatalf("Failed to create client: %v", err)
    }

    ctx := context.Background()

    // Get current log size
    sth, err := logClient.GetSTH(ctx)
    if err != nil {
        log.Fatalf("Failed to get STH: %v", err)
    }

    // Get last scanned index from database
    var lastIndex int64
    err = db.QueryRow("SELECT last_index FROM log_progress WHERE log_url = ?",
        logURL).Scan(&lastIndex)
    if err == sql.ErrNoRows {
        lastIndex = 0
    } else if err != nil {
        log.Fatal(err)
    }

    fmt.Printf("Log has %d entries, last scanned: %d\n", sth.TreeSize, lastIndex)

    if uint64(lastIndex) >= sth.TreeSize {
        fmt.Println("No new entries")
        return
    }

    // Scan new entries
    opts := scanner.ScannerOptions{
        Matcher:    &scanner.MatchAll{},
        NumWorkers: 2,
        FetcherOptions: scanner.FetcherOptions{
            StartIndex: lastIndex,
            EndIndex:   int64(sth.TreeSize) - 1,
            BatchSize:  100,
        },
    }

    s := scanner.NewScanner(logClient, opts)

    processedCount := 0
    err = s.Scan(ctx,
        func(entry *ct.RawLogEntry) {
            processedCount++
            // Process certificate entry
            logEntry, _ := entry.ToLogEntry()
            if logEntry != nil && logEntry.X509Cert != nil {
                fmt.Printf("New cert: %s\n", logEntry.X509Cert.Subject)
            }
        },
        func(entry *ct.RawLogEntry) {
            processedCount++
            // Process precertificate entry
        },
    )

    if err != nil {
        log.Fatalf("Scan failed: %v", err)
    }

    // Update last scanned index
    _, err = db.Exec(`
        INSERT OR REPLACE INTO log_progress (log_url, last_index, last_update)
        VALUES (?, ?, ?)
    `, logURL, sth.TreeSize-1, time.Now())

    if err != nil {
        log.Fatal(err)
    }

    fmt.Printf("Processed %d new entries\n", processedCount)
}
```

## Working with Log Lists

### Loading and Searching Log Lists

Parse the Chrome/Google log list and find specific logs.

```go
package main

import (
    "encoding/json"
    "fmt"
    "log"
    "net/http"
    "time"

    "github.com/google/certificate-transparency-go/loglist3"
)

func main() {
    // Fetch the official Chrome CT log list
    httpClient := &http.Client{Timeout: 30 * time.Second}

    resp, err := httpClient.Get(loglist3.LogListURL)
    if err != nil {
        log.Fatalf("Failed to fetch log list: %v", err)
    }
    defer resp.Body.Close()

    // Parse log list
    var llData []byte
    llData, err = io.ReadAll(resp.Body)
    if err != nil {
        log.Fatalf("Failed to read log list: %v", err)
    }

    logList, err := loglist3.NewFromJSON(llData)
    if err != nil {
        log.Fatalf("Failed to parse log list: %v", err)
    }

    fmt.Printf("Log List Version: %s\n", logList.Version)
    fmt.Printf("Published: %v\n", logList.LogListTimestamp)
    fmt.Printf("Total operators: %d\n\n", len(logList.Operators))

    // Find Google-operated logs
    for _, operator := range logList.Operators {
        if operator.GoogleOperated() {
            fmt.Printf("Google Operator: %s\n", operator.Name)
            fmt.Printf("  Email: %v\n", operator.Email)
            fmt.Printf("  Logs: %d\n", len(operator.Logs))

            for _, ctLog := range operator.Logs {
                fmt.Printf("    - %s\n", ctLog.Description)
                fmt.Printf("      URL: %s\n", ctLog.URL)
                fmt.Printf("      MMD: %d seconds\n", ctLog.MMD)
                fmt.Printf("      State: %s\n", ctLog.State)
            }
            fmt.Println()
        }
    }

    // Find a specific log by name
    logs := logList.FindLogByName("argon")
    fmt.Printf("\nFound %d logs matching 'argon':\n", len(logs))
    for _, ctLog := range logs {
        fmt.Printf("  %s: %s\n", ctLog.Description, ctLog.URL)
    }

    // Find log by URL
    targetURL := "https://ct.googleapis.com/logs/us1/argon2024/"
    foundLog := logList.FindLogByURL(targetURL)
    if foundLog != nil {
        fmt.Printf("\nLog details for %s:\n", targetURL)
        fmt.Printf("  Description: %s\n", foundLog.Description)
        fmt.Printf("  Log ID: %x\n", foundLog.LogID)
        fmt.Printf("  MMD: %d seconds\n", foundLog.MMD)

        if foundLog.TemporalInterval != nil {
            fmt.Printf("  Valid for certs: %v to %v\n",
                foundLog.TemporalInterval.StartInclusive,
                foundLog.TemporalInterval.EndExclusive)
        }
    }
}
```

### Filtering Logs by State

Filter logs based on their operational state (usable, qualified, retired, etc.).

```go
package main

import (
    "fmt"
    "log"

    "github.com/google/certificate-transparency-go/loglist3"
)

func main() {
    // Load log list (from file or HTTP - see previous example)
    logList, err := loadLogList() // Implementation omitted for brevity
    if err != nil {
        log.Fatal(err)
    }

    // Count logs by state
    stateCounts := make(map[loglist3.LogStatus]int)
    usableLogs := make([]*loglist3.Log, 0)

    for _, operator := range logList.Operators {
        for _, ctLog := range operator.Logs {
            status := ctLog.State.LogStatus()
            stateCounts[status]++

            if status == loglist3.UsableLogStatus {
                usableLogs = append(usableLogs, ctLog)
            }
        }
    }

    fmt.Println("Log State Summary:")
    for state, count := range stateCounts {
        fmt.Printf("  %s: %d\n", state.String(), count)
    }

    fmt.Printf("\nUsable logs for submission (%d):\n", len(usableLogs))
    for _, ctLog := range usableLogs {
        fmt.Printf("  %s\n", ctLog.Description)
        fmt.Printf("    URL: %s\n", ctLog.URL)

        if ctLog.TemporalInterval != nil {
            fmt.Printf("    Temporal: %v - %v\n",
                ctLog.TemporalInterval.StartInclusive,
                ctLog.TemporalInterval.EndExclusive)
        }
        fmt.Println()
    }
}

func loadLogList() (*loglist3.LogList, error) {
    // Implementation to load log list from file or HTTP
    return nil, nil
}
```

## CT Policy Enforcement

### Chrome CT Policy Validation

Validate that a certificate has sufficient SCTs according to Chrome's CT policy.

```go
package main

import (
    "context"
    "fmt"
    "log"
    "time"

    ct "github.com/google/certificate-transparency-go"
    "github.com/google/certificate-transparency-go/ctpolicy"
    "github.com/google/certificate-transparency-go/loglist3"
    "github.com/google/certificate-transparency-go/x509"
)

func main() {
    // Load certificate and log list
    cert := loadCertificate() // Implementation omitted
    logList := loadLogList()   // Implementation omitted

    // Create Chrome CT policy
    policy := ctpolicy.ChromeCTPolicy{}

    // Get log groups required for this certificate
    groups, err := policy.LogsByGroup(cert, logList)
    if err != nil {
        log.Fatalf("Failed to get log groups: %v", err)
    }

    fmt.Printf("Certificate: %s\n", cert.Subject)
    fmt.Printf("Lifetime: %d months\n", lifetimeMonths(cert))
    fmt.Println("\nRequired log submissions:")

    for groupName, group := range groups {
        fmt.Printf("\nGroup: %s\n", groupName)
        fmt.Printf("  Minimum SCTs required: %d\n", group.MinInclusions)
        fmt.Printf("  Available logs: %d\n", len(group.LogURLs))

        if group.IsBase {
            fmt.Printf("  Type: Base group (all logs)\n")
        }
    }

    // Simulate checking if we have enough SCTs
    receivedSCTs := map[string]*ct.SignedCertificateTimestamp{
        "https://ct.googleapis.com/logs/us1/argon2024/": &ct.SignedCertificateTimestamp{},
        "https://ct.googleapis.com/logs/eu1/xenon2024/": &ct.SignedCertificateTimestamp{},
    }

    // Check policy compliance
    compliant := true
    for groupName, group := range groups {
        matchCount := 0
        for logURL := range receivedSCTs {
            if group.LogURLs[logURL] {
                matchCount++
            }
        }

        if matchCount < group.MinInclusions {
            compliant = false
            fmt.Printf("\n❌ Group '%s': Only %d/%d required SCTs\n",
                groupName, matchCount, group.MinInclusions)
        } else {
            fmt.Printf("\n✓ Group '%s': %d/%d required SCTs\n",
                groupName, matchCount, group.MinInclusions)
        }
    }

    if compliant {
        fmt.Println("\n✓ Certificate meets Chrome CT Policy")
    } else {
        fmt.Println("\n❌ Certificate does NOT meet Chrome CT Policy")
    }
}

func lifetimeMonths(cert *x509.Certificate) int {
    start := cert.NotBefore
    end := cert.NotAfter

    years := end.Year() - start.Year()
    months := int(end.Month()) - int(start.Month())
    lifetimeMonths := years*12 + months

    if end.Day() < start.Day() {
        lifetimeMonths--
    }
    return lifetimeMonths
}

func loadCertificate() *x509.Certificate { return nil }
func loadLogList() *loglist3.LogList { return nil }
```

### Multi-Log Submission with Policy

Submit certificates to multiple logs to satisfy CT policy requirements.

```go
package main

import (
    "context"
    "fmt"
    "log"
    "net/http"
    "sync"
    "time"

    ct "github.com/google/certificate-transparency-go"
    "github.com/google/certificate-transparency-go/client"
    "github.com/google/certificate-transparency-go/ctpolicy"
    "github.com/google/certificate-transparency-go/jsonclient"
    "github.com/google/certificate-transparency-go/loglist3"
    "github.com/google/certificate-transparency-go/x509"
)

type SCTResult struct {
    LogURL string
    SCT    *ct.SignedCertificateTimestamp
    Error  error
}

func main() {
    // Load certificate chain and log list
    chain := []ct.ASN1Cert{
        {Data: /* leaf cert DER */},
        {Data: /* issuer cert DER */},
    }
    cert := loadCertificate() // Parsed x509.Certificate
    logList := loadLogList()

    // Create policy and get required log groups
    policy := ctpolicy.ChromeCTPolicy{}
    groups, err := policy.LogsByGroup(cert, logList)
    if err != nil {
        log.Fatalf("Failed to get log groups: %v", err)
    }

    // Build list of target logs from groups
    targetLogs := make(map[string]bool)
    for _, group := range groups {
        session := group.GetSubmissionSession()
        // Take enough logs to satisfy the group requirement
        for i := 0; i < group.MinInclusions && i < len(session); i++ {
            targetLogs[session[i]] = true
        }
    }

    fmt.Printf("Submitting to %d logs...\n", len(targetLogs))

    // Submit to logs in parallel
    httpClient := &http.Client{Timeout: 30 * time.Second}
    ctx, cancel := context.WithTimeout(context.Background(), 60*time.Second)
    defer cancel()

    resultsCh := make(chan SCTResult, len(targetLogs))
    var wg sync.WaitGroup

    for logURL := range targetLogs {
        wg.Add(1)
        go func(url string) {
            defer wg.Done()

            logClient, err := client.New(url, httpClient, jsonclient.Options{})
            if err != nil {
                resultsCh <- SCTResult{LogURL: url, Error: err}
                return
            }

            sct, err := logClient.AddChain(ctx, chain)
            resultsCh <- SCTResult{LogURL: url, SCT: sct, Error: err}
        }(logURL)
    }

    // Close results channel when all submissions complete
    go func() {
        wg.Wait()
        close(resultsCh)
    }()

    // Collect results
    scts := make(map[string]*ct.SignedCertificateTimestamp)
    failures := make([]string, 0)

    for result := range resultsCh {
        if result.Error != nil {
            fmt.Printf("❌ %s: %v\n", result.LogURL, result.Error)
            failures = append(failures, result.LogURL)
        } else {
            fmt.Printf("✓ %s: SCT received\n", result.LogURL)
            scts[result.LogURL] = result.SCT
        }
    }

    // Check if we still meet policy requirements
    for groupName, group := range groups {
        matchCount := 0
        for logURL := range scts {
            if group.LogURLs[logURL] {
                matchCount++
            }
        }

        if matchCount < group.MinInclusions {
            log.Printf("Warning: Group '%s' has only %d/%d required SCTs",
                groupName, matchCount, group.MinInclusions)
        }
    }

    fmt.Printf("\nSubmission complete: %d SCTs received, %d failures\n",
        len(scts), len(failures))
}

func loadCertificate() *x509.Certificate { return nil }
func loadLogList() *loglist3.LogList { return nil }
```

## Utility Functions

### Calculate Leaf Hash from Certificate and SCT

Compute the Merkle tree leaf hash for certificate-SCT pairs.

```go
package main

import (
    "encoding/base64"
    "fmt"
    "log"

    ct "github.com/google/certificate-transparency-go"
    "github.com/google/certificate-transparency-go/ctutil"
    "github.com/google/certificate-transparency-go/x509"
)

func main() {
    // Load certificate chain
    cert := loadCertificate() // Implementation omitted
    issuer := loadIssuer()     // Implementation omitted
    chain := []*x509.Certificate{cert, issuer}

    // SCT received from log
    sct := &ct.SignedCertificateTimestamp{
        SCTVersion: ct.V1,
        Timestamp:  1704067200000,
        LogID:      ct.LogID{KeyID: [32]byte{ /* log ID */ }},
        Extensions: ct.CTExtensions{},
        Signature:  ct.DigitallySigned{ /* signature data */ },
    }

    // Calculate leaf hash for X.509 certificate
    leafHash, err := ctutil.LeafHash(chain, sct, false)
    if err != nil {
        log.Fatalf("Failed to calculate leaf hash: %v", err)
    }

    fmt.Printf("Leaf Hash (hex): %x\n", leafHash)

    // Get base64 encoding for get-proof-by-hash API
    leafHashB64, err := ctutil.LeafHashB64(chain, sct, false)
    if err != nil {
        log.Fatalf("Failed to encode leaf hash: %v", err)
    }

    fmt.Printf("Leaf Hash (base64): %s\n", leafHashB64)
    fmt.Println("\nUse this base64 value with get-proof-by-hash API:")
    fmt.Printf("GET /ct/v1/get-proof-by-hash?hash=%s&tree_size=<size>\n", leafHashB64)

    // For certificates with embedded SCTs, use embedded=true
    if len(cert.SCTList.SCTList) > 0 {
        embeddedHash, err := ctutil.LeafHash(chain, sct, true)
        if err != nil {
            log.Fatalf("Failed to calculate embedded leaf hash: %v", err)
        }
        fmt.Printf("\nEmbedded SCT Leaf Hash: %x\n", embeddedHash)
    }
}

func loadCertificate() *x509.Certificate { return nil }
func loadIssuer() *x509.Certificate { return nil }
```

### Working with Precertificates

Handle precertificates and poison extensions.

```go
package main

import (
    "crypto/sha256"
    "fmt"
    "log"

    ct "github.com/google/certificate-transparency-go"
    "github.com/google/certificate-transparency-go/x509"
)

func main() {
    // Load precertificate and issuer
    precert := loadPrecertificate() // Implementation omitted
    issuer := loadIssuer()          // Implementation omitted

    // Check if certificate is a precertificate
    if precert.IsPrecertificate() {
        fmt.Println("✓ This is a precertificate")

        // Check for poison extension
        hasPoisonExt := false
        for _, ext := range precert.Extensions {
            if ext.Id.Equal(x509.OIDExtensionCTPoison) {
                hasPoisonExt = true
                if !ext.Critical {
                    log.Println("Warning: Poison extension is not critical")
                }
                break
            }
        }

        if !hasPoisonExt {
            log.Println("Warning: No poison extension found")
        }
    }

    // Check if issuer is a precertificate issuer (has CT EKU)
    isPreIssuer := ct.IsPreIssuer(issuer)
    if isPreIssuer {
        fmt.Println("Issuer is a precertificate signing certificate")
    }

    // Create Merkle tree leaf for precertificate
    timestamp := uint64(1704067200000)
    chain := []*x509.Certificate{precert, issuer}

    leaf, err := ct.MerkleTreeLeafFromChain(chain, ct.PrecertLogEntryType, timestamp)
    if err != nil {
        log.Fatalf("Failed to create leaf: %v", err)
    }

    // The leaf will contain the defanged TBS (without poison extension)
    if leaf.TimestampedEntry.PrecertEntry != nil {
        issuerKeyHash := leaf.TimestampedEntry.PrecertEntry.IssuerKeyHash
        fmt.Printf("Issuer Key Hash: %x\n", issuerKeyHash)

        tbsLength := len(leaf.TimestampedEntry.PrecertEntry.TBSCertificate)
        fmt.Printf("TBS Certificate Length: %d bytes\n", tbsLength)

        // Verify issuer key hash
        expectedHash := sha256.Sum256(issuer.RawSubjectPublicKeyInfo)
        if issuerKeyHash == expectedHash {
            fmt.Println("✓ Issuer key hash matches")
        } else {
            fmt.Println("❌ Issuer key hash mismatch")
        }
    }

    // Calculate leaf hash
    leafHash, err := ct.LeafHashForLeaf(leaf)
    if err != nil {
        log.Fatalf("Failed to calculate leaf hash: %v", err)
    }

    fmt.Printf("Precertificate Leaf Hash: %x\n", leafHash)
}

func loadPrecertificate() *x509.Certificate { return nil }
func loadIssuer() *x509.Certificate { return nil }
```

## Command Line Tools

### Using ctclient for Log Interaction

The ctclient command-line tool provides easy access to CT log operations.

```bash
# Get the signed tree head from a log
ctclient get-sth --log_uri="https://ct.googleapis.com/logs/us1/argon2024/"

# Get entries from a log (indices 100-105)
ctclient get-entries \
  --log_uri="https://ct.googleapis.com/logs/us1/argon2024/" \
  --first=100 \
  --last=105

# Get a consistency proof between two tree sizes
ctclient get-consistency-proof \
  --log_uri="https://ct.googleapis.com/logs/us1/argon2024/" \
  --first=1000 \
  --second=2000

# Get an inclusion proof for a specific leaf hash
ctclient get-inclusion-proof \
  --log_uri="https://ct.googleapis.com/logs/us1/argon2024/" \
  --leaf_hash="x5YxB3AwG7WEEPGm3q8p8h3h6qH5D2qGFG3Q+qH5D2o=" \
  --tree_size=1000000

# Upload a certificate chain to get an SCT
ctclient upload \
  --log_uri="https://ct.googleapis.com/logs/us1/argon2024/" \
  --chain="cert.pem" \
  --chain="issuer.pem"

# Get accepted root certificates
ctclient get-roots \
  --log_uri="https://ct.googleapis.com/logs/us1/argon2024/"

# Bisect a log to find when an entry was added
ctclient bisect \
  --log_uri="https://ct.googleapis.com/logs/us1/argon2024/" \
  --timestamp=1704067200000
```

### Using sctcheck to Verify SCTs

Verify SCT signatures using the sctcheck utility.

```bash
# Verify an SCT from a PEM-encoded certificate
sctcheck \
  --cert_file="certificate.pem" \
  --sct_file="certificate.sct" \
  --log_key="log_public_key.pem"

# Verify embedded SCTs in a certificate
sctcheck \
  --cert_file="certificate_with_embedded_scts.pem" \
  --log_key="log_public_key.pem" \
  --embedded

# Verify SCT with explicit log list
sctcheck \
  --cert_file="certificate.pem" \
  --sct_file="certificate.sct" \
  --log_list="https://www.gstatic.com/ct/log_list/v3/log_list.json"

# Check all SCTs in a certificate against known logs
sctcheck \
  --cert_file="certificate.pem" \
  --log_list="https://www.gstatic.com/ct/log_list/v3/log_list.json" \
  --check_inclusion
```

### Using scanlog to Scan Logs

Scan CT logs for certificates matching specific criteria.

```bash
# Scan a log for all entries
scanlog \
  --log_uri="https://ct.googleapis.com/logs/us1/argon2024/" \
  --start_index=0 \
  --end_index=10000

# Scan for certificates matching a domain pattern
scanlog \
  --log_uri="https://ct.googleapis.com/logs/us1/argon2024/" \
  --cert_regex=".*\\.example\\.com" \
  --start_index=0

# Scan for certificates from a specific CA
scanlog \
  --log_uri="https://ct.googleapis.com/logs/us1/argon2024/" \
  --issuer_regex="Let's Encrypt" \
  --parallel_fetch=4 \
  --batch_size=100

# Output matching certificates to a file
scanlog \
  --log_uri="https://ct.googleapis.com/logs/us1/argon2024/" \
  --cert_regex=".*\\.example\\.com" \
  --output_file="matches.json" \
  --json_output

# Scan only precertificates
scanlog \
  --log_uri="https://ct.googleapis.com/logs/us1/argon2024/" \
  --precert_only \
  --start_index=0 \
  --end_index=50000
```

## Summary

This Go library provides a complete, production-ready implementation of Certificate Transparency suitable for both client-side verification and server-side log operation. The core client library enables interaction with RFC 6962 CT logs through HTTP and DNS protocols, supporting all standard operations including certificate submission, SCT retrieval, STH queries, and cryptographic proof verification. The scanner package facilitates large-scale log monitoring with configurable parallelism and custom matching logic, making it practical to process millions of log entries efficiently.

The library's policy enforcement modules implement Chrome and Apple CT requirements, automatically calculating required log submissions based on certificate lifetime and ensuring compliance with browser policies. The Trillian-based CT log server implementation provides a scalable backend for operating production CT logs, with support for multiple log instances, quota management, and integration with Trillian's distributed architecture. Additional tools for working with precertificates, embedded SCTs, log lists, and certificate chain validation make this a comprehensive toolkit for any Certificate Transparency workflow, from simple SCT verification to running enterprise-scale monitoring systems.