Neurotex – Architecting safer software with AI
Abstract
Neurotex is a plugin-based software architecture analysis tool that inspects open-source repositories for security and architectural risks. It utilizes LLMs for reasoning over code comments, README files, and configurations to infer potential vulnerabilities and architectural flaws. By integrating Retrieval-Augmented Generation (RAG), it provides suggestions aligned with OWASP, NIST, and RSC standards. With core quality attributes like extensibility, security, and modularity, the system supports…
Author
Moksh Mehta - s4826878
Functionality
The complete version of LLM Power Analyzer will offer the following features:
- Input a GitHub repository URL or upload a project as a ZIP file.
- Analyze the project’s structure and generate a Software Bill of Materials (SBOM).
- Detect insecure or vulnerable code patterns using rule-based and plugin-based scanners.
- Use LLM-based summarization and reasoning to uncover architectural risks.
- Leverage RAG to provide insights based on security standards (OWASP, NIST, RSC).
- Generate detailed vulnerability and architecture reports with severity levels and recommendations.
- Support plugins for custom scanners, reports, or LLM tasks.
- Provide a minimal UI for web-based initiation alongside a CLI tool.
Scope
Minimum Viable Product (MVP) will include:
- GitHub repo or ZIP file input.
- Basic dependency and logic pattern scanning.
- LLM-based vulnerability reasoning.
- A basic plugin system with at least one custom plugin.
- Simple CLI or static HTML UI.
- Export of basic vulnerability and architectural risk report.
Quality Attributes
Extensibility
Definition: The ease with which new functionalities (e.g. plugins or modules) can be added without impacting existing system components.
Justification: As security threats evolve, the tool must quickly adopt new scanners or reasoning models without deep refactoring.
Measurement: Measured by time taken and code changes required to add a new plugin (e.g. a Dockerfile scanner).
Security
Definition: The system’s ability to detect and resist misuse, attacks, or data leakage.
Justification: Security analysis is core to the tool’s functionality and also necessary in the tool’s own implementation.
Measurement: Use prompt injection test cases, malicious repos, and leak detection tests to validate robustness.
Modularity
Definition: The degree to which the system’s components are separated into independent, replaceable units.
Justification: A modular structure allows for easier maintenance, testing, and upgrades.
Measurement: Evaluated using software metrics (e.g. cyclomatic complexity, test coverage, file separation), plus architectural diagrams showing module boundaries.
Evaluation
Extensibility Evaluation
- Attempt to introduce a new plugin (e.g. Dockerfile vulnerability scanner) mid-project.
- Assess how independently it can run alongside the core logic.
- Measure the changes required in the core system to integrate this new plugin.
Security Evaluation
- Use prompt injection test cases to evaluate LLM robustness.
- Analyze detection rates using a mix of benign and malicious open-source repositories.
- Perform controlled tests for memory leaks and LLM memorization vulnerabilities.
Modularity Evaluation
- Run static analysis for code complexity and module test coverage.
- Ensure clear file separation and functional boundaries in the system.
- Present a visual diagram illustrating architectural components and plugin interfaces.