Automated threat model updates with GitHub Actions

A threat model is only useful if it stays current. When the codebase changes, the threat model should reflect those changes. This guide shows how to automate that with GitHub Actions and an AI coding agent.

The concept

The .justappsec file format is designed for both humans and AI. Humans interact through the JustAppSec editor. AI agents work with the JSON directly.

Each file contains an embedded $schema field with full instructions for AI agents - the structure, valid values, and update rules. This makes the file self-documenting. Any AI reading it knows exactly how to update it correctly without needing external documentation.

This workflow fits into existing protected branch processes. When someone opens a PR with code changes, the pipeline:

1. Reviews the changes in context of the full codebase
2. Updates the threat model to reflect the changes
3. Commits the update to the PR branch
4. Adds a comment summarising what changed

The threat model update becomes part of the normal PR review. No new PRs, no extra approval steps - just an additional commit that reviewers see alongside the code.

GitHub Actions with OpenAI Codex

Codex CLI runs as a proper agent inside the repository. It can explore files, understand the codebase structure, and make informed decisions about what to update. This is far more capable than sending prompts to the API.

Note: This workflow only runs on PRs from branches within the same repository. PRs from forks do not receive secrets and cannot push commits back to the source branch. For open source projects with external contributors, consider running the update manually or using a workflow_dispatch trigger after the PR is merged.

# .github/workflows/update-threat-model.yml
name: Update threat model

on:
  pull_request:
    types: [opened, synchronize]
    paths:
      - 'src/**'
      - 'app/**'
      - 'lib/**'

jobs:
  update-threat-model:
    runs-on: ubuntu-latest
    permissions:
      contents: write
      pull-requests: write

    steps:
      - uses: actions/checkout@v4
        with:
          ref: ${{ github.event.pull_request.head.ref }}
          fetch-depth: 0

      - name: Check if this is a threat model update commit
        id: check-skip
        run: |
          if git log -1 --pretty=%B | grep -q '\[threat-model-auto\]'; then
            echo "skip=true" >> $GITHUB_OUTPUT
            echo "Skipping - this commit was made by the threat model workflow"
          else
            echo "skip=false" >> $GITHUB_OUTPUT
          fi

      - name: Setup Node
        if: steps.check-skip.outputs.skip != 'true'
        uses: actions/setup-node@v4
        with:
          node-version: '20'

      - name: Install Codex CLI
        if: steps.check-skip.outputs.skip != 'true'
        run: npm install -g @openai/codex

      - name: Update threat model
        if: steps.check-skip.outputs.skip != 'true'
        id: update
        env:
          OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
        run: |
          codex --exec "You are reviewing a pull request and updating the threat model.

          PRIMARY TASK: Review the changes in this PR and update threat-model.justappsec.

          CONTEXT:
          - This PR is on branch ${{ github.head_ref }} targeting ${{ github.base_ref }}
          - The specific commit is ${{ github.event.pull_request.head.sha }}

          INSTRUCTIONS:
          1. Read threat-model.justappsec to understand the current threat model
          2. Explore the codebase to understand the full system architecture
          3. Review the changes in this PR - trace data flows from source to sink
          4. Also consider recent changes in the repo that may not yet be reflected in the threat model
          5. Update threat-model.justappsec:
             - Add new threats if changes introduce security-relevant functionality
             - Update existing threats affected by the changes
             - Close threats where mitigations are now in place
          
          OUTPUT: Write the updated threat model to threat-model.justappsec. Must be valid JSON matching the embedded schema."

      - name: Validate and commit
        if: steps.check-skip.outputs.skip != 'true'
        id: commit
        run: |
          CHANGED_FILES=$(git diff --name-only)
          
          if [ -z "$CHANGED_FILES" ]; then
            echo "updated=false" >> $GITHUB_OUTPUT
            echo "No threat model changes needed"
            exit 0
          fi
          
          if [ "$CHANGED_FILES" != "threat-model.justappsec" ]; then
            echo "::warning::Agent modified files other than threat-model.justappsec. Skipping."
            git checkout -- .
            echo "updated=false" >> $GITHUB_OUTPUT
            exit 0
          fi
          
          if ! python3 -c "import json; json.load(open('threat-model.justappsec'))" 2>/dev/null; then
            echo "::warning::Agent produced invalid JSON. Skipping."
            git checkout -- threat-model.justappsec
            echo "updated=false" >> $GITHUB_OUTPUT
            exit 0
          fi
          
          # Count changes for the PR comment
          ADDED=$(git diff threat-model.justappsec | grep -c '^+.*"id":' || true)
          REMOVED=$(git diff threat-model.justappsec | grep -c '^-.*"id":' || true)
          echo "added=$ADDED" >> $GITHUB_OUTPUT
          echo "removed=$REMOVED" >> $GITHUB_OUTPUT
          
          git config user.name "github-actions[bot]"
          git config user.email "github-actions[bot]@users.noreply.github.com"
          git add threat-model.justappsec
          git commit -m "Update threat model [threat-model-auto]"
          git push
          echo "updated=true" >> $GITHUB_OUTPUT

      - name: Comment on PR
        if: steps.commit.outputs.updated == 'true'
        uses: actions/github-script@v7
        with:
          script: |
            const added = '${{ steps.commit.outputs.added }}';
            const removed = '${{ steps.commit.outputs.removed }}';
            
            let summary = '**Threat model updated**\n\n';
            summary += 'Reviewed the code changes and updated `threat-model.justappsec`.\n\n';
            if (added !== '0') summary += `- ${added} threat(s) added or modified\n`;
            if (removed !== '0') summary += `- ${removed} threat(s) removed or closed\n`;
            summary += '\nPlease review the threat model changes alongside the code.';
            
            github.rest.issues.createComment({
              owner: context.repo.owner,
              repo: context.repo.repo,
              issue_number: context.issue.number,
              body: summary
            });

The [threat-model-auto] marker in the commit message prevents re-triggering. The workflow commits directly to the PR branch - the threat model update becomes part of the PR for reviewers to assess alongside the code changes.

Using Claude Code

Claude Code runs as an agentic process with full access to the repository. It can read any file, trace code paths, and write the updated threat model directly.

- name: Install Claude Code
  run: npm install -g @anthropic-ai/claude-code

- name: Update threat model with Claude
  env:
    ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
  run: |
    claude-code "You are reviewing a pull request and updating the threat model.

    PRIMARY TASK: Review the changes in this PR and update threat-model.justappsec.

    CONTEXT:
    - This PR is on branch ${{ github.head_ref }} targeting ${{ github.base_ref }}
    - The specific commit is ${{ github.event.pull_request.head.sha }}

    INSTRUCTIONS:
    1. Read threat-model.justappsec to understand the current threat model
    2. Explore the codebase to understand the full system architecture
    3. Review the changes in this PR - trace data flows from source to sink
    4. Also consider recent changes in the repo that may not yet be reflected in the threat model
    5. Update threat-model.justappsec:
       - Add new threats if changes introduce security-relevant functionality
       - Update existing threats affected by the changes
       - Close threats where mitigations are now in place
    
    OUTPUT: Write the updated threat model to threat-model.justappsec. Must be valid JSON."

Use the same validation and commit pattern as the Codex example - check for the [threat-model-auto] marker to skip re-runs, verify only the threat model file was modified, and add a PR comment summarising the changes.

Verification step

For higher confidence, add a verification pass. Use a second agent invocation to review the changes:

- name: Verify updates
  env:
    OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
  run: |
    TM_DIFF=$(git diff HEAD~1 threat-model.justappsec)
    
    codex --exec "Review these threat model changes and verify they are accurate:
    
    $TM_DIFF
    
    Check that:
    - New threats reference real code paths in this repo
    - Closed threats are actually mitigated in the code
    - Severity levels match the actual impact
    
    If the changes look correct, output just 'APPROVED'. Otherwise describe the specific issues."

Or use a different model for the verification pass to get a second opinion:

- name: Verify with Claude
  env:
    ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
  run: |
    claude-code "Review the changes to threat-model.justappsec (git diff HEAD~1). 
    Verify each change against the actual codebase. Flag anything that looks wrong."

Triggering on specific changes

You may not want to run this on every PR. Filter to security-relevant paths:

on:
  pull_request:
    paths:
      - 'src/auth/**'
      - 'src/api/**'
      - 'lib/crypto/**'
      - 'app/api/**'
      - '**/middleware.*'

Or trigger manually when needed:

on:
  workflow_dispatch:
    inputs:
      reason:
        description: 'Reason for update'
        required: true

The result

When someone opens a PR that changes authentication logic, the pipeline:

1. Reviews the changes in context of the full codebase
2. Updates the threat model to reflect new risks or mitigations
3. Validates the JSON
4. Commits the update to the PR branch
5. Comments on the PR with a summary

The reviewer sees both the code change and the threat model change in the same PR. Security thinking stays in sync with the code.