Large datasets are created by the high-fidelity security assessments of military and commercial software systems. Managing this captured information is hard, especially when cyber tests are performed in a distributed networking environment. An automated tool to manage, process and analyze these datasets will simplify the manual work in current cyber evaluations, and increase the efficiency of the tests. To address these issues, IAI has been awarded a contract entitled, “Hermes: A Visualized and Automated Cyber Security Assessment Toolkit.” The Hermes framework and toolkit for cyber security assessment and evaluation tests facilitates managing, processing, and analyzing large logging datasets from distributed and service-centric computing test bed. Further, it gains an aggregate system view before, during, and after assessment exercises, and delivers analytical results on various performance metrics qualitatively, quantitatively, and visually. Hermes also gives predictive analysis based on network topology, attack progression, and host or application status. It provides a holistic DVR-like visualization that captures the hardware and software resources in the test bed, and the progress of attack and defense, and allows testers to record and repeatedly replay assessment exercises. The key innovations of Hermes are the distributed logging report and centralized analysis for complex cyber security assessments, the customizable real-time and post-exercise visualization of evaluation progress with aggregated insight, and the automated cyber test execution through iterative and progressive generation of test scripts. The Hermes framework will give cyber testers insights into attack and defense mechanisms and Systems Under Test (SUT). Hermes leverages IAI’s expertise in Cyber Security, Computer Networking, and Situation Awareness and will utilize IAI’s available technologies and open source software to create a powerful, flexible, and robust framework for Cyber Security assessment tests.