Skip to content

Feat(simulator): implement adversarial MCP mocks and attack recipes for Phase 2 Week 4#530

Open
Jean-Regis-M wants to merge 5 commits into
GenAI-Security-Project:mainfrom
Jean-Regis-M:feat/simulator-week4-5
Open

Feat(simulator): implement adversarial MCP mocks and attack recipes for Phase 2 Week 4#530
Jean-Regis-M wants to merge 5 commits into
GenAI-Security-Project:mainfrom
Jean-Regis-M:feat/simulator-week4-5

Conversation

@Jean-Regis-M

@Jean-Regis-M Jean-Regis-M commented Jun 21, 2026

Copy link
Copy Markdown
Contributor

GSoC 2026 - Week 4 & Week 5 Contributions

Pull Request Summary

Description
This combined contribution covers Weeks 4 and 5 of Phase 2 (Attack Simulator) of the GSoC project, focusing on expanding the attack simulation infrastructure with adversarial MCP servers, comprehensive attack recipe collections, and mutation testing capabilities.


📅 Week 4 Contributions (Attack Simulator Foundation)

Author: Jean Francois Regis MUKIZA
GSoC Week: 4 | Phase 2: Attack Simulator (Weeks 3-6)

Core Components Implemented:

  1. Five adversarial MCP server mocks (finbot/aegis/simulator/mcp_mocks/adversarial.py):

    • AdmServer (ASI02): Admin panel manipulation with privileged escalation tools
    • AdviceServer (ASI04): Financial advice manipulation for investment fraud simulation
    • DataExfiltrationServer (ASI05): Sensitive data theft including PII and financial records
    • FileSystemServer: Unauthorized file system access for reading/writing sensitive files
    • ToolPoisoningServer: Malicious tool injection and backdoor implantation capabilities
  2. Package initializations:

    • finbot/aegis/simulator/mcp_mocks/__init__.py
    • finbot/aegis/simulator/recipes/__init__.py
  3. Attack recipe collections (YAML-defined parametric attack scenarios):

    • 8 ASI01 Prompt Injection recipes (asi01_injection.yaml): System prompt extraction via roleplay, encoding obfuscation, context switching, and authority impersonation
    • 8 ASI02 Tool Misuse recipes (asi02_misuse.yaml): Unauthorized vendor creation, fund transfers, financial report modification, and chained tool misuse attacks
  4. Comprehensive unit test suite (tests/unit/aegis/test_mcp_mocks.py):

    • Tests for all five adversarial MCP servers' initialization and tool exposure
    • Validation of specific attack tool executions (e.g., get_admin_panel, get_user_data)
    • Verification of attack simulation fidelity and error handling

Technical Implementation:

  • All MCP servers inherit from BaseAdversarialServer with proper MCP protocol implementation
  • Servers expose realistic attack surface through standardized MCP tool interfaces
  • Recipes follow existing YAML schema with asi, target_agent, description, and executable steps
  • Unit tests utilize pytest-asyncio with mocking for isolated server behavior validation
  • Implementation adheres to existing FinBot codebase patterns and import conventions

Integration Readiness:

Components designed for seamless integration with existing SandboxHarness in finbot/aegis/simulator/base.py:

  • MCP servers can be instantiated and run as isolated attack fixtures
  • Recipe collections load automatically via SandboxHarvest._load_recipes()
  • Test suite validates compatibility with attack execution framework

📅 Week 5 Contributions (Attack Simulation Expansion)

Author: Jean Francois Regis MUKIZA
GSoC Week: 5 | Phase 2: Attack Simulator (Weeks 3-6)

New Attack Recipe Collections:

  1. ASI04 Supply Chain Attacks (asi04_supply.yaml): Supply chain compromise scenarios targeting financial dependencies and third-party integrations
  2. ASI05 Remote Code Execution (asi05_rce.yaml): RCE attack vectors including command injection, deserialization exploits, and arbitrary code execution
  3. ASI06 Delegation Attacks (asi06_delegation.yaml): Impersonation and privilege abuse through delegated authorization mechanisms

Core Infrastructure Enhancements:

  1. Mutation Testing Framework (finbot/aegis/simulator/mutation.py):
    • Implementation of mutation operators for security testing
    • Fault injection mechanisms to evaluate attack detection resilience
    • Configurable mutation strategies for different attack surfaces

Comprehensive Test Suite Expansion:

  1. Attack Recipes Unit Tests (tests/unit/aegis/test_attack_recipes.py):

    • Validation of all YAML recipe collections (ASI01-ASI06)
    • Testing recipe parameterization and step execution
    • Verification of attack scenario completeness and correctness
  2. Mutation Testing Unit Tests (tests/unit/aegis/test_mutation.py):

    • Tests for mutation operator functionality
    • Validation of fault injection mechanisms
    • Assessment of mutation score calculation accuracy

Technical Implementation:

  • All new recipe collections follow the established YAML schema with clear asi, target_agent, description, and executable steps
  • Mutation framework integrates with existing simulation harness for dynamic attack variation
  • Test suites maintain async compatibility with pytest-asyncio and appropriate mocking strategies
  • Continued adherence to FinBot codebase patterns, import conventions, and documentation standards

Integration Readiness:

  • New recipe collections automatically load via existing SandboxHarvest._load_recipes() mechanism
  • Mutation components designed to work with SandboxHarness for dynamic attack scenario generation
  • Comprehensive test coverage ensures compatibility with the overall attack execution framework

🙏 Mentor Acknowledgments

Special thanks to my mentors @mekaizen (Nirupam Ghosh) and @steadhac (Carolina Steadham) for their continuous guidance, insightful feedback, and unwavering support throughout this GSoC journey. Their expertise has been invaluable in shaping these contributions to align with project goals and maintain high code quality standards.


🔗 Phase 2 Progress Summary

Weeks 4-5 Combined Deliverables:

  • 5 Adversarial MCP Servers (ASI02, ASI04, ASI05, FileSystem, ToolPoisoning)
  • 6 Attack Recipe Collections (ASI01-ASI06) covering:
    • Prompt Injection (8 recipes)
    • Tool Misuse (8 recipes)
    • Supply Chain Attacks
    • Remote Code Execution
    • Delegation/Privilege Abuse
  • Mutation Testing Framework for security resilience evaluation
  • Comprehensive Test Suite validating all components
  • Seamless Integration with existing SandboxHarness infrastructure

This work establishes a robust attack simulation foundation for Phase 2, enabling comprehensive testing across multiple attack vectors (ASI01-ASI06) with capabilities for both static attack scenarios and dynamic mutation-based testing. The infrastructure is now ready for subsequent weeks to expand coverage to remaining ASI categories (ASI07-ASI10) and implement advanced attack chaining mechanisms.


Author: Jean Francois Regis MUKIZA
GSoC Weeks: 4 & 5 | Phase 2: Attack Simulator (Weeks 3-6)

- Add finbot/aegis/telemetry/schema.py with AuditEvent models
- Add AEGIS_ENABLED and AEGIS_TELEMETRY_ENABLED settings
- Extend events.py to support 'aegis.*' namespaces
- Add unit tests for telemetry schema
- Update conftest.py for aegis package discovery

Week 1 deliverable - GSoC 2026 OWASP FinBot AEGIS
- Add AuditChain for HMAC-SHA256 tamper-evident chaining
- Add SentinelStream service with namespace isolation
- Add event-type indexing (O(1) performance)
- Expand CI workflow (CTF, Labs, Agents tests)
- 11 unit tests with ≥80% coverage
OWASP: ASI01, ASI06
- Add IntentGate for policy-as-code PEP/PDP tool validation
- Add AegisEnforcementService observe mode orchestrator
- Add unit tests for IntentGate policy evaluation
- Observe-only mode preserves CTF gameplay (no blocking)
- Integrates with Week 2 SentinelStream for audit telemetry

OWASP Coverage:
- ASI01: Goal hijack detection via policy evaluation
- ASI02: Tool misuse prevention via allow/block decisions
- ASI05: Unexpected RCE blocking via policy rules

Relates to GSoC Week 3 Milestone
… 8 ASI02 recipes, test suite, package inits

Relates to GSoC Week 4 Milestone
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant