Monitoring Tools and Techniques for Software Supply Chain Security¶

The Role of Monitoring in Supply Chain Security¶

Supply chain attacks often remain undetected for extended periods. Without proper monitoring, malicious code injections, unauthorized access, and other security breaches can persist silently, compromising numerous systems downstream. Effective monitoring provides:

graph TD
    classDef benefit fill:#3498db, stroke:#333, stroke-width:1px, color:white
    classDef component fill:#2ecc71, stroke:#333, stroke-width:1px
    classDef attack fill:#e74c3c, stroke:#333, stroke-width:1px, color:white

    A[Supply Chain<br/>Monitoring]:::component

    A --> B[Early Attack<br/>Detection]:::benefit
    A --> C[Build Process<br/>Integrity]:::benefit
    A --> D[Dependency<br/>Verification]:::benefit
    A --> E[Provenance<br/>Tracking]:::benefit
    A --> F[Anomaly<br/>Detection]:::benefit

    B --> B1[Repository<br/>Compromises]:::attack
    B --> B2[Credential<br/>Theft]:::attack

    C --> C1[Build Server<br/>Tampering]:::attack
    C --> C2[Pipeline<br/>Injection]:::attack

    D --> D1[Malicious<br/>Dependencies]:::attack
    D --> D2[Typosquatting<br/>Attacks]:::attack

    E --> E1[Artifact<br/>Tampering]:::attack

    F --> F1[Unusual Access<br/>Patterns]:::attack
    F --> F2[Abnormal Build<br/>Behavior]:::attack

    click B "#early-detection-systems" "View early detection systems"
    click F "#anomaly-detection" "View anomaly detection systems"

    style A stroke-width:3px

Monitoring Domains for Supply Chain Security¶

3. Dependency Ecosystem Monitoring¶

• Dependency changes: Additions, removals, or updates to dependencies
• Dependency behaviors: Runtime activities of third-party components
• Package registry events: Unusual updates or ownership transfers
• Vulnerability disclosures: New CVEs affecting your dependency ecosystem
• Maintainer activities: Unusual behavior from key project maintainers

4. Artifact & Deployment Monitoring¶

• Artifact integrity: Verification that artifacts match expected source code builds
• Signature validation: Continuous verification of digital signatures
• Deployment patterns: Changes in how and when artifacts are deployed
• Runtime behaviors: Unexpected behaviors from deployed applications
• Infrastructure changes: Modifications to deployment environments

Early Detection Systems¶

Repository & Code Monitoring Tools¶

Implementation Example: Repository Monitoring¶

# GitHub Actions workflow for repository monitoring
name: Repository Security Monitoring

on:
  schedule:
    - cron: '0 */6 * * *'  # Every 6 hours
  push:
    branches: [ main, develop ]

jobs:
  monitor-repo:
    name: Repository Security Scan
    runs-on: ubuntu-latest
    steps:
      - name: Checkout repository
        uses: actions/checkout@v3
        with:
          fetch-depth: 0

      # Detect unexpected branch protection changes
      - name: Check branch protections
        uses: fregante/protected-branch-check-action@v1
        with:
          main: main
          pull-request-required: true
          reviewers-required: 2
          status-checks-required: true

      # Secret scanning with advanced patterns
      - name: Scan for secrets
        uses: gitleaks/gitleaks-action@v2
        with:
          config-path: .gitleaks.toml

      # Detect unusual commit patterns
      - name: Analyze commit patterns
        run: |
          echo "Analyzing commit patterns..."
          # Custom script to analyze commit patterns
          # - Time-of-day anomalies
          # - Author behavior changes
          # - Unusual file modifications
          python .github/scripts/analyze_commits.py

      # Report findings
      - name: Post findings to security dashboard
        if: ${{ success() || failure() }}
        run: |
          curl -X POST ${{ secrets.SECURITY_DASHBOARD_URL }} \
            -H "Authorization: Bearer ${{ secrets.DASHBOARD_TOKEN }}" \
            -H "Content-Type: application/json" \
            --data-binary @report.json

Build System Monitoring¶

# Example build integrity monitoring script

#!/bin/bash
# This script monitors build process integrity

# 1. Verify build script hasn't changed
BUILD_SCRIPT_HASH=$(sha256sum build.sh | cut -d ' ' -f 1)
EXPECTED_HASH="e9c5682857d70e097b8a914688f9cddb...."

if [ "$BUILD_SCRIPT_HASH" != "$EXPECTED_HASH" ]; then
  echo "ERROR: Build script has been modified!"
  exit 1
fi

# 2. Verify build environment
if ! docker info &> /dev/null; then
  echo "ERROR: Docker not available!"
  exit 1
fi

# 3. Capture build provenance
BUILD_ID=$(uuidgen)
BUILD_TIME=$(date -u +"%Y-%m-%dT%H:%M:%SZ")
BUILD_USER=$(whoami)
BUILD_HOST=$(hostname)
GIT_COMMIT=$(git rev-parse HEAD)

# 4. Record network connections during build
echo "Starting network monitoring..."
tcpdump -i any -w build_network_$BUILD_ID.pcap &
TCPDUMP_PID=$!

# 5. Run the build with resource monitoring
time docker build -t myapp:$BUILD_ID .
BUILD_EXIT=$?

# 6. Stop monitoring
kill $TCPDUMP_PID

# 7. Generate attestation
cat > attestation.json << EOL
{
  "buildId": "$BUILD_ID",
  "timestamp": "$BUILD_TIME",
  "builder": "$BUILD_USER@$BUILD_HOST",
  "sourceCommit": "$GIT_COMMIT",
  "exitCode": $BUILD_EXIT
}
EOL

# 8. Sign attestation
gpg --detach-sign --armor attestation.json

echo "Build provenance recorded in attestation.json"

Build System Monitoring Tools¶

Dependency Monitoring¶

Key Monitoring Approaches¶

1. Vulnerability Monitoring
2. Continuous scanning: Regular rescanning of dependencies against vulnerability databases
3. Alert integration: Notification systems for newly discovered vulnerabilities
4. Exploit monitoring: Tracking exploitation status of vulnerabilities in dependencies

5. Patch notification: Alerts when fixes become available

6. Behavioral Monitoring

7. Runtime behavior analysis: Detecting when dependencies act unexpectedly
8. Network activity: Monitoring unexpected connections from third-party code
9. Filesystem access: Watching for unusual file operations

10. Resource consumption: Identifying abnormal resource usage patterns

11. Ecosystem Intelligence

12. Maintainer activity: Changes in project ownership or maintainer behavior
13. Package popularity: Unusual changes in download patterns
14. Update frequency: Unexpected release patterns or update timing
15. Repository activities: Monitoring for suspicious commits or pull requests

Implementation Example: Continuous Dependency Monitoring¶

# Example: Proactive dependency monitoring script
# This script continuously monitors for new vulnerabilities in dependencies

#!/bin/bash
set -e

# Directory to store vulnerability data
VULN_DIR="./vulnerability-data"
mkdir -p $VULN_DIR

# Function to check dependencies
check_dependencies() {
  echo "Scanning dependencies at $(date)"

  # For npm projects
  if [ -f package.json ]; then
    echo "Scanning npm dependencies..."
    npm audit --json > $VULN_DIR/npm-audit-$(date +%F).json

    # Compare with previous results
    if [ -f "$VULN_DIR/npm-audit-previous.json" ]; then
      NEW_VULNS=$(jq -r 'if .vulnerabilities then .vulnerabilities | keys | length else 0 end' $VULN_DIR/npm-audit-$(date +%F).json)
      OLD_VULNS=$(jq -r 'if .vulnerabilities then .vulnerabilities | keys | length else 0 end' $VULN_DIR/npm-audit-previous.json)

      if [ $NEW_VULNS -gt $OLD_VULNS ]; then
        echo "⚠️ WARNING: New vulnerabilities detected!"
        jq -r '.vulnerabilities | keys | .[]' $VULN_DIR/npm-audit-$(date +%F).json

        # Send notification
        curl -X POST $WEBHOOK_URL \
          -H "Content-Type: application/json" \
          -d "{\"text\":\"New vulnerabilities detected in npm dependencies!\"}"
      fi
    fi

    # Save current state as previous for next run
    cp $VULN_DIR/npm-audit-$(date +%F).json $VULN_DIR/npm-audit-previous.json
  fi

  # For Java projects
  if [ -f pom.xml ]; then
    echo "Scanning Java dependencies..."
    mvn org.owasp:dependency-check-maven:check -DoutputDirectory=$VULN_DIR

    # Process results and send alerts similarly...
  fi
}

# Main monitoring loop
while true; do
  check_dependencies
  echo "Sleeping for 6 hours..."
  sleep 21600  # Check every 6 hours
done

Anomaly Detection Systems¶

Implementation Example: Build Behavior Anomaly Detection¶

# Example Python code for build anomaly detection

import pandas as pd
import numpy as np
from sklearn.ensemble import IsolationForest
import matplotlib.pyplot as plt
import json

# Load historical build data
def load_build_data(filename):
    with open(filename, 'r') as f:
        data = json.load(f)

    builds = []
    for build in data:
        builds.append({
            'duration': build['duration'],
            'file_count': build['output_files'],
            'binary_size': build['output_size_bytes'],
            'cpu_usage': build['avg_cpu_percent'],
            'memory_usage': build['max_memory_mb'],
            'network_calls': build['external_network_calls'],
            'exit_code': build['exit_code']
        })

    return pd.DataFrame(builds)

# Create anomaly detection model
def train_model(df):
    model = IsolationForest(
        contamination=0.05,  # Expect ~5% anomalies
        random_state=42
    )

    # Train on numerical features
    features = ['duration', 'file_count', 'binary_size', 
                'cpu_usage', 'memory_usage', 'network_calls']
    model.fit(df[features])

    return model, features

# Detect anomalies in new builds
def detect_anomalies(model, features, historical_df, new_build):
    # Convert new build to DataFrame with same structure
    new_df = pd.DataFrame([new_build])

    # Make prediction (1: normal, -1: anomaly)
    prediction = model.predict(new_df[features])[0]

    if prediction == -1:
        # Find what's anomalous about this build
        anomalies = []
        for feature in features:
            mean = historical_df[feature].mean()
            std = historical_df[feature].std()
            value = new_build[feature]

            # If more than 3 standard deviations away, flag as anomalous
            if abs(value - mean) > 3 * std:
                anomalies.append({
                    'feature': feature,
                    'value': value,
                    'mean': mean,
                    'std_dev': std,
                    'std_deviations': abs(value - mean) / std
                })

        return True, anomalies

    return False, []

# Main function
def main():
    # Load historical build data
    historical_data = load_build_data('build_history.json')

    # Train anomaly detection model
    model, features = train_model(historical_data)

    # New build to check
    new_build = {
        'duration': 128,
        'file_count': 1892,
        'binary_size': 15650432,
        'cpu_usage': 82,
        'memory_usage': 1245,
        'network_calls': 17,  # Unusual, typically 0-2 for this build
        'exit_code': 0
    }

    # Detect anomalies
    is_anomaly, anomalies = detect_anomalies(model, features, historical_data, new_build)

    if is_anomaly:
        print("Anomalous build detected!")
        for anomaly in anomalies:
            print(f"Unusual {anomaly['feature']}: {anomaly['value']} "
                 f"(typical: {anomaly['mean']:.2f} ± {anomaly['std_dev']:.2f}, "
                 f"{anomaly['std_deviations']:.2f} standard deviations away)")
    else:
        print("Build appears normal.")

if __name__ == "__main__":
    main()

Security Information and Event Management (SIEM)¶

Key SIEM Features for Supply Chain Security¶

1. Data Collection and Aggregation
2. Log collection from development, build, and deployment systems
3. API integration with code repositories, CI/CD platforms, and cloud services
4. SBOM and vulnerability data ingestion

5. Authentication and access control event collection

6. Correlation and Analysis

7. Cross-domain event correlation (e.g., repository activity with build events)
8. Behavior analytics to identify suspicious patterns
9. Rule-based detection of known attack patterns

10. Machine learning for anomaly detection

11. Visualization and Reporting

12. Supply chain security dashboards
13. Attack path visualization
14. Compliance reporting
15. Security metrics and KPIs

Popular SIEM Solutions for Supply Chain Security¶

Splunk Query Example for Supply Chain Security Monitoring¶

/* Detecting potential supply chain attacks by correlating
   unusual repository activity with build anomalies */

/* Step 1: Find unusual repository events */
index=git sourcetype=github 
| stats count by user, repo, action, time_window=1h 
| eventstats avg(count) as avg_count, stdev(count) as stdev_count by user, action
| where count > avg_count + (3 * stdev_count)
| rename user as suspicious_user, repo as affected_repo
| table _time, suspicious_user, affected_repo, action, count

/* Step 2: Find related build events */
| join affected_repo 
    [search index=cicd sourcetype=jenkins 
    | where repository=affected_repo
    | stats list(build_id) as builds by repository
    | table repository, builds]

/* Step 3: Examine artifacts produced by suspicious builds */
| join builds 
    [search index=cicd sourcetype=artifacts 
    | rename build_id as artifact_build_id
    | where match(builds, artifact_build_id)
    | table artifact_build_id, artifact_name, artifact_hash]

/* Step 4: Check if artifacts were deployed */
| join artifact_name
    [search index=deployments 
    | rename artifact as deployed_artifact
    | where deployed_artifact=artifact_name
    | table deployed_artifact, deployment_environment, deployment_time]

/* Final correlation showing potential attack path */
| table _time, suspicious_user, affected_repo, action, 
         builds, artifact_name, artifact_hash, 
         deployment_environment, deployment_time
| sort -_time

Runtime Monitoring¶

# Example: Falco rules for detecting supply chain attacks at runtime

# Detect unexpected network connections from build processes
- rule: Unexpected Network Connection from Build Process
  desc: Detect when a build process makes unexpected network connections
  condition: >
    spawned_process and
    proc.name in (make, mvn, gradle, npm, pip) and
    (outbound_connection and not 
     dest.ip in (allowed_build_destinations))
  output: >
    Network connection from build tool (user=%user.name
    command=%proc.cmdline connection=%fd.name)
  priority: WARNING
  tags: [supply-chain, build-process, network]

# Detect modifications to running containers
- rule: Container Modification
  desc: Detect writes to container file systems at runtime
  condition: >
    container and
    (open_write or create or mkdir or rename) and
    not proc.name in (allowed_write_processes) and
    not fd.directory in (allowed_write_directories)
  output: >
    File system write in container (user=%user.name
    command=%proc.cmdline file=%fd.name container=%container.name)
  priority: WARNING
  tags: [supply-chain, container, runtime]

# Detect execution of unusual binaries in production
- rule: Unusual Binary Execution
  desc: Detect execution of binaries not part of base image
  condition: >
    container and
    spawned_process and
    not proc.name in (allowed_processes) and
    not proc.name startswith ("docker-")
  output: >
    Unusual binary execution (user=%user.name
    command=%proc.cmdline container=%container.name)
  priority: WARNING
  tags: [supply-chain, container, runtime]

Comprehensive Monitoring Architecture¶

A mature supply chain monitoring architecture integrates multiple monitoring domains to provide comprehensive visibility and detection capabilities:

flowchart TD
    classDef source fill:#3498db, stroke:#333, stroke-width:1px, color:white
    classDef build fill:#e74c3c, stroke:#333, stroke-width:1px, color:white
    classDef deploy fill:#2ecc71, stroke:#333, stroke-width:1px
    classDef runtime fill:#f39c12, stroke:#333, stroke-width:1px
    classDef central fill:#9b59b6, stroke:#333, stroke-width:1px, color:white

    subgraph "Source Monitoring"
        A1[Repository Activity<br>Monitoring]:::source
        A2[Secrets Detection]:::source
        A3[Access Control<br>Monitoring]:::source
    end

    subgraph "Build Monitoring"
        B1[Build Server<br>Security]:::build
        B2[Pipeline<br>Monitoring]:::build
        B3[Artifact<br>Verification]:::build
    end

    subgraph "Deployment Monitoring"
        C1[Image Registry<br>Monitoring]:::deploy
        C2[Deployment<br>Authorization]:::deploy
        C3[Configuration<br>Management]:::deploy
    end

    subgraph "Runtime Monitoring"
        D1[Container<br>Runtime Security]:::runtime
        D2[Application<br>Behavior Analysis]:::runtime
        D3[Infrastructure<br>Monitoring]:::runtime
    end

    subgraph "Central Analysis"
        E1[SIEM<br>Platform]:::central
        E2[Threat<br>Intelligence]:::central
        E3[Anomaly<br>Detection]:::central
        E4[Correlation<br>Engine]:::central
    end

    A1 --> E1
    A2 --> E1
    A3 --> E1
    B1 --> E1
    B2 --> E1
    B3 --> E1
    C1 --> E1
    C2 --> E1
    C3 --> E1
    D1 --> E1
    D2 --> E1
    D3 --> E1

    E1 --> E2
    E1 --> E3
    E2 --> E4
    E3 --> E4
    E4 --> F[Security<br>Operations]
    F --> G[Incident<br>Response]

Implementing a Supply Chain Monitoring Strategy¶

1. Assessment and Planning¶

• Supply Chain Mapping: Document your entire supply chain from development to deployment
• Risk Assessment: Identify the most critical points in your supply chain
• Monitoring Goals: Define specific monitoring objectives and requirements
• Tool Evaluation: Assess tools based on your specific needs and environment

2. Implementation Phases¶

3. Best Practices for Monitoring Implementation¶

Incident Response Integration¶

Key Integration Points¶

1. Alert Triage and Escalation
2. Automated triage based on alert severity and context
3. Clear escalation paths for different alert types

4. On-call rotations for critical supply chain alerts

5. Playbook Integration

6. Predefined response playbooks for common supply chain attacks
7. Automated playbook triggering based on alert types

8. Regular testing and refinement of response procedures

9. Containment Automation

10. Automated containment actions for critical threats
11. Integration with CI/CD systems to halt affected pipelines

12. Ability to revoke compromised credentials or access

13. Root Cause Investigation

14. Comprehensive forensics data collection
15. Timeline reconstruction capabilities
16. Threat hunting tools for extended investigation

Example: Supply Chain Attack Response Playbook¶

sequenceDiagram
    participant M as Monitoring System
    participant S as SIEM
    participant IR as Incident Response Team
    participant B as Build System
    participant CR as Container Registry
    participant D as Deployment System

    Note over M,D: Supply Chain Compromise Detection & Response

    M->>S: Anomaly detected in build process
    S->>S: Correlate with other security events
    S->>IR: Generate high priority alert

    IR->>S: Investigate alert details

    alt Confirmed Compromise
        IR->>B: Pause affected build pipelines
        IR->>CR: Quarantine suspicious artifacts
        IR->>D: Block deployment of affected artifacts

        IR->>IR: Initiate forensic investigation
        IR->>IR: Identify affected systems & artifacts
        IR->>IR: Determine root cause

        IR->>B: Rebuild from verified source
        IR->>CR: Release clean artifacts
        IR->>D: Deploy verified artifacts

        IR->>IR: Document incident & update detections
    else False Positive
        IR->>S: Document false positive
        IR->>S: Update detection rules
    end

Conclusion¶

• Cover the entire supply chain from source code to runtime environments
• Integrate multiple monitoring domains for comprehensive visibility
• Combine signature-based and behavioral detection for maximum effectiveness
• Focus on anomaly detection to identify sophisticated and novel attacks
• Provide context-rich alerting to enable rapid investigation and response
• Integrate with incident response processes for seamless remediation

By implementing a robust monitoring strategy, organizations can significantly reduce the risk of successful supply chain attacks and minimize the impact of security incidents that do occur.

Additional Resources¶

• CISA Guidance on Defending Against Software Supply Chain Attacks
• NIST SP 800-161: Supply Chain Risk Management Practices
• Supply Chain Integrity Model (SCIM)
• Cloud Native Security Monitoring
• Google's BeyondProd: Cloud-Native Security for Software Supply Chains