Cybersecurity training company Hack The Box Ltd. today introduced AI Range Enterprise Edition to let security teams test whether their own artificial intelligence agents can do the jobs they have been assigned.
The release is seeking to take on a problem in how security teams vet their agents. Hack The Box says an agent may be put into production after a strong benchmark result or vendor test without ever being checked against the work it will handle day to day. Human staff in the same role usually are. An agent’s performance can also drift after deployment as the model underneath it gets updated.
Hack The Box launched the original AI Range last December. Since then, AI labs, government customers and hyperscalers have been testing agents on it, according to the company. Corporate security teams get the same testing with the enterprise edition, and each agent is appraised against the role it has been given.
The appraisal method goes by the name Agentic Worker Competence. Each agent gets a score for its assigned role and a pass-or-fail mark in each environment it works through, with results tracked over time. New environments are added on a regular basis as fresh vulnerabilities and attack patterns turn up in the wild. Customers can rerun the appraisal whenever an agent, its model or its data changes.
Chief Product Officer Gerasimos Marketos summed up the process as “connect the agent, assign it a role and let it work.” The platform then logs how the agent copes with each task on its own, he said, and security leaders can use those results to decide what work an agent is ready for.
The launch also covers the human staff who work alongside agents. Leaders need “confidence in both sides of the workforce” as AI takes on more cybersecurity tasks, according to founder and Chief Executive Haris Pylarinos.
Penetration tester and security operations center analyst are the only two roles with AI-augmented versions at launch. In the analyst version, practitioners lean on AI to review alerts in realistic scenarios. Whatever the AI recommends doing next is theirs to evaluate.
Grading in those roles runs through what Hack The Box calls Agentic Operator Competence Scoring. The scenarios test whether a practitioner catches errors in an agent’s reasoning and knows when to step in. Holding back is scored as well, based on whether someone lets AI-driven work run when it does not need correcting.
AI Range Enterprise Edition and the two AI-augmented roles are available now, with more cybersecurity roles planned for the coming months.
Hack The Box is a venture capital-backed startup that raised $55 million in a Series B round led by The Carlyle Group Inc. in 2023, with Paladin Capital Group and Marathon Venture Capital also participating. The company says it has more than 800 enterprise customers and a community of more than 4 million cybersecurity professionals.