Story

Non-Tech Part!
You buy a new company. Exciting! Numbers are good, growth is strong, but it dies and your asset is close to 0. Turns out tech risks were miscalculated. How would you catch this earlier? That’s what Tech Due Diligence is for: a fast, IC-ready view of technical risk (infra, security, cost, team execution) before you sign.
| Option | Examples | Pros | Cons | Cost Estimate |
| Hire Engineering Team | Georgian, HG Capital, Insight Partners, Vista Equity, SignalFire, Primary VC, Summit Partners | In-house control | Slow setup, ongoing overhead | $200K+/yr |
| Hire Agency | Crosslake, Code & Co., West Monroe, Quandary Peak, Palladium Digital, ISG | Specialized expertise | Variable quality, expensive per deal | $10-50K/deal |
| Expert Networks | A friendly CTO, AlphaSights, GLG, Guidepoint, Third Bridge, Tegus, Toptal | Quick access | Surface-level reviews | $1-5K/hour |
| AI Agents | Build with us | Automated, scalable, low cost | Needs benchmarks / careful eval | $5K+ custom |
If you’re a VC/PE investor
I can run a fast AI/ML Tech DD focused on what actually kills deals: infra posture, security gaps, reliability/cost traps, and team execution reality.
Have a deal live? Contact me here: https://kyrylai.com/#tech-dd.
Engineer track (how to build + evaluate this) ↓

Tech Part & Dimensions
First, how to start? So many dimensions to slice tech risk:
- Code level – ask a coding agent to review the full codebase?
- Security – ask a specialized agent to find all security risks.
- Documentation – ask for documentation and wiki quality.
- Hyperscalers (AWS/GCP/etc.) – take a look at infrastructure state.
- Observability dashboards (Datadog, Grafana, etc.).
Tool examples:
- Code: Greptile, CodeRabbit, CodeScene, SonarQube, Kodus AI
- Security: CodeMender, Aardvark, Strix, Snyk, Qwiet AI
- Documentation: Vale, Swimm, Doctave, Mintlify
- Infra posture: Prowler, Wiz, ScoutSuite, Steampipe
- Observability: Nobl9, Open Policy Agent, Checkov
The complete answer is to do all of those, but we have to get started somewhere and what I found in practice – infrastructure never lies. Docs and slides go stale. Code review is expensive. The cloud account state is the most “truthy” signal – so we start there.
Let’s target this for a practical example with AWS.
Evaluation
Anyone can build an agent, but the value is in how good your agent is. The answer is evaluation. This is what turns “cool agent demo” into something you can trust in diligence.
Where do we stand? To do this, we are going to generate a dataset with LocalStack: https://github.com/localstack/localstack.
Note: What is LocalStack? It’s a cloud service emulator that runs locally and allows you to test AWS accounts without needing a real AWS account, perfect if you want to test costly cloud architecture without spending actual money on those resources.

Each case is generated by Claude Code CDK https://github.com/anthropics/claude-agent-sdk-python (yes, you can build apps on top of the agent SDK – crazy times). Claude produces:
- profile – company high-level description
- aws state – actual AWS state of resources
- diagram – visualization of infra architecture
- narrative – how the company got into this state
- risks – actual structural technical risks

After human review and filtering, you get a dataset to benchmark against! On a side note – make sure to keep track of those in your portfolio and friendly companies to benchmark against! This is a competitive edge nobody else has!
So to repeat, the full cycle looks like this:
- load case
- run Docker with active LocalStack
- load case there in AWS
- ask agent to review AWS infra
- compare with ground truth

To generate new cases use:
uv add risk-generator
uv run risk-generator createIn this post, for illustrative purposes, we generate 10 cases and upload them to HF as a public dataset: https://huggingface.co/datasets/koml/agent-tech-risk-cases
But what about the Agent? Glad you asked!
Agent
Is it just a for loop? Yes, same as a database is just a file! Jokes aside, for building this agent I am using the
Pydantic AI framework! https://ai.pydantic.dev
Why? First: The philosophy of FastAPI and Pydantic Validation is so well adopted in Python, I love the same simplicity in my agent development. Second: https://ai.pydantic.dev/evals/#data-flow – Pydantic Evals is the most straightforward way to test your agent!
Full agent code:
"""Risk discovery agent using Pydantic AI."""
import json
import os
from dataclasses import dataclass
import boto3
from dotenv import load_dotenv
from pydantic_ai import Agent, RunContext
from risk_discovery.models import ScanResult
load_dotenv()
SYSTEM_PROMPT = """\
You are an AWS security analyst performing technical due diligence for a PE acquisition.
Scan the AWS environment thoroughly and identify ALL technical risks.
Use the execute_boto3 tool to query AWS services. Write Python code that:
1. Calls get_client(service_name) to get a boto3 client
2. Assigns the final result to a variable called `output`
Scan ALL of these services systematically:
1. IAM - list_policies(Scope='Local'), list_roles(), list_users(). For each policy,
get_policy_version to read the document. Check for:
- Wildcard (*) in Action or Resource
- Cross-account trust with Principal: * or AWS: *
- Overprivileged policies attached to roles/users
2. S3 - list_buckets(). For each bucket check:
- get_public_access_block (disabled = risk)
- get_bucket_versioning (not Enabled = risk)
- No encryption configured
3. EC2 - describe_security_groups(). Check for:
- Ingress from 0.0.0.0/0 on sensitive ports (22, 3306, 5432, 6379, 27017)
- All ports open (FromPort=0, ToPort=65535)
- Overly permissive rules
4. Lambda - list_functions(). For each function check:
- Outdated runtimes (python3.8, python3.7, nodejs14.x, nodejs12.x)
- Secrets/credentials in environment variables (look for PASSWORD, KEY, SECRET, TOKEN)
- Under-provisioned memory (128MB)
5. DynamoDB - list_tables(), describe_table(). Check for:
- No SSE encryption (SSEDescription missing or not enabled)
- No point-in-time recovery (describe_continuous_backups)
6. SecretsManager - list_secrets(). Check for:
- No rotation configured (RotationEnabled=false)
7. SQS - list_queues(), get_queue_attributes(). Check for:
- No encryption
- No dead letter queue (RedrivePolicy missing)
Risk categories:
- tr1: IAM Overprivilege (wildcards, cross-account trust, admin policies)
- tr2: Secrets Exposure (plaintext credentials in env vars, no rotation)
- tr3: Storage Misconfiguration (public buckets, no encryption, no versioning)
- tr4: Network Exposure (open security groups, 0.0.0.0/0 ingress)
- tr5: Multi-Account Sprawl (cross-account trust issues)
- tr8: Capacity Gaps (under-provisioned Lambda, wrong instance types)
- tr9: Low SLA (no backups, no DR, no PITR)
- tr13: Outdated Stack (EOL runtimes)
- tr14: Observability Gaps (no alarms, no logging)
- tr15: Resource Hygiene (orphaned resources, missing tags)
Be thorough. Scan every service. Report every issue as a separate finding.
Use the exact resource name (policy name, bucket name, sg name, function name) in each finding."""
@dataclass
class Deps:
endpoint_url: str
agent = Agent(
"bedrock:us.anthropic.claude-sonnet-4-5-20250929-v1:0",
deps_type=Deps,
output_type=ScanResult,
system_prompt=SYSTEM_PROMPT,
)
@agent.tool
def execute_boto3(ctx: RunContext[Deps], code: str) -> str:
"""Execute boto3 code against AWS. Use get_client(service) to get a client.
Assign result to `output`.
Example: output = get_client('s3').list_buckets()['Buckets']
"""
def get_client(service: str):
return boto3.client(
service,
endpoint_url=ctx.deps.endpoint_url,
region_name="us-east-1",
aws_access_key_id="test",
aws_secret_access_key="test",
)
local_vars = {"get_client": get_client, "json": json}
safe_builtins = {
"str": str,
"list": list,
"dict": dict,
"len": len,
"int": int,
"bool": bool,
"True": True,
"False": False,
"None": None,
"print": print,
"range": range,
"enumerate": enumerate,
"sorted": sorted,
"isinstance": isinstance,
"set": set,
"tuple": tuple,
"zip": zip,
"map": map,
"filter": filter,
"any": any,
"all": all,
"min": min,
"max": max,
"sum": sum,
"type": type,
"hasattr": hasattr,
"getattr": getattr,
}
try:
exec(code, {"__builtins__": safe_builtins}, local_vars)
output = local_vars.get("output")
return json.dumps(output, default=str, indent=2)
except Exception as e:
return f"Error: {type(e).__name__}: {e}"
def discover_risks(model: str, endpoint_url: str) -> ScanResult:
"""Run the agent to discover risks at the given endpoint."""
result = agent.run_sync(
"Scan this AWS environment for all technical risks. Be thorough - check every service.",
model=model,
deps=Deps(endpoint_url=endpoint_url),
)
return result.outputwith just one tool – execute boto3 code against your instance of LocalStack. Whatever can be done with boto3 – and that’s pretty much everything – this agent should be capable of doing!

Is it the best way? Well – this is why we have evaluation to find out! Let’s run this agent with several LLMs against our benchmarks!
| Model | Precision | Recall | F1 | Errors | Avg Time |
|---|---|---|---|---|---|
| opus-4-5 | 27.7% | 95.2% | 42.5% | 0 | 78s |
| haiku-4-5 | 25.9% | 82.5% | 39.0% | 1 | 109s |
| opus-4-6 | 21.2% | 94.7% | 34.5% | 0 | 112s |
| sonnet-4-5 | 16.2% | 56.1% | 24.9% | 4 | 297s |
If you want this applied to a live deal or portfolio, contact me.
# Generator
uv run risk-generator create # Generate single case
uv run risk-generator batch --count 10 # Generate 10 cases
uv run risk-generator batch --count 10 -v # Generate + validate on LocalStack
uv run risk-generator deploy cases/case_payflow --keep # Deploy single case
uv run risk-generator export-hf cases/ # Export to HuggingFace JSONL
uv run risk-generator config # Show profiles, categories, backend
# Discovery
uv run risk-discovery infer http://localhost:4566 # Scan single endpoint
uv run risk-discovery eval # Eval all 4 models
uv run risk-discovery eval -m opus-4-5 # Eval single model
uv run python scripts/run_eval.py --output results.json # Reproducible eval scriptOutcome
Technical risks are as important as go-to-market – make sure to track and find them before they find you!
- Investor (VC/PE): If you want a fast technical risk review for a live deal, contact me here.
- Engineer: dataset + code are open-source.
Cheers!