Editor Reviews:

Product Description:
<>The Definitive Guide to Quantifying, Classifying, and Measuring Enterprise IT Security Operations

Security Metrics is the first comprehensive best-practice guide to defining, creating, and utilizing security metrics in the enterprise.

Using sample charts, graphics, case studies, and war stories, Yankee Group Security Expert Andrew Jaquith demonstrates exactly how to establish effective metrics based on your organization’s unique requirements. You’ll discover how to quantify hard-to-measure security activities, compile and analyze all relevant data, identify strengths and weaknesses, set cost-effective priorities for improvement, and craft compelling messages for senior management.

Security Metrics successfully bridges management’s quantitative viewpoint with the nuts-and-bolts approach typically taken by security professionals. It brings together expert solutions drawn from Jaquith’s extensive consulting work in the software, aerospace, and financial services industries, including new metrics presented nowhere else. You’ll learn how to:

• Replace nonstop crisis response with a systematic approach to security improvement

• Understand the differences between “good” and “bad” metrics

• Measure coverage and control, vulnerability management, password quality, patch latency, benchmark scoring, and business-adjusted risk

• Quantify the effectiveness of security acquisition, implementation, and other program activities

• Organize, aggregate, and analyze your data to bring out key insights

• Use visualization to understand and communicate security issues more clearly

• Capture valuable data from firewalls and antivirus logs, third-party auditor reports, and other resources

• Implement balanced scorecards that present compact, holistic views of organizational security effectiveness

Whether you’re an engineer or consultant responsible for security and reporting to management–or an executive who needs better information for decision-making–Security Metrics is the resource you have been searching for.

Andrew Jaquith, program manager for Yankee Group’s Security Solutions and Services Decision Service, advises enterprise clients on prioritizing and managing security resources. He also helps security vendors develop product, service, and go-to-market strategies for reaching enterprise customers. He co-founded @stake, Inc., a security consulting pioneer acquired by Symantec Corporation in 2004. His application security and metrics research has been featured in CIO, CSO, InformationWeek, IEEE Security and Privacy, and The Economist.

Foreword         

Preface            

Acknowledgments         

About the Author           

Chapter 1          Introduction: Escaping the Hamster Wheel of Pain          

Chapter 2          Defining Security Metrics           

Chapter 3          Diagnosing Problems and Measuring Technical Security  

Chapter 4          Measuring Program Effectiveness           

Chapter 5          Analysis Techniques     

Chapter 6          Visualization     

Chapter 7          Automating Metrics Calculations

Chapter 8          Designing Security Scorecards  

Index   

Rating : - Some gaps, but useful nonetheless
Andrew Jaquith's book on security metrics is refreshing in its approach. Instead of a neverending cycle of risk assessments and vulnerability patching (a process which the author humorously calls the "hamster wheel of pain"), we are told to focus on core operational security processes and measurement of key indicators.

The central premise of the book is that a "risk management" approach, as promoted by many security vendors, doesn't work. The reason it doesn't work is that it is extremely difficult to get a good handle on the true value of assets, and an accurate estimate of risk. As the author puts it, "identifying problems is easy ... quantifying and valuing risk is much harder."

The thorough discussion of information security metrics makes this book worthwhile reading. However, there is a hint of sloppy thinking sprinkled throughout, which tends to undermine one's trust in the author's intellectual honesty. For example, when discussing the importance of tracking not only inbound viruses, but outbound as well, the author makes the following statement:

BEGIN QUOTE -
Another twist I have added to the traditional antivirus statistics is a simple metric documenting the number of outbound viruses or spyware samples caught by the perimeter mail gateway's content filtering software. Why it matters is simple--it is an excellent indicator of how "clean" the internal network is. Organizations that practice good hygiene don't infect their neighbors and business partners. My friend Dan Geer relates this quote from the CSO of a Wall Street investment bank:

"Last year we stopped 70,000 inbound viruses, but I am prouder of having stopped 500 outbound."

In other words, the bank's internal network is cleaner than the outside environment by a factor of 140 to 1.
- END QUOTE

Certainly, the conclusion in the last sentence cannot be supported without additional information. The volume of inbound email is likely to be drastically higher, which may account for the difference. The bank's outbound detection/prevention mechanism also may not be as efficient as the inbound.

Moreover, the metrics analysis chapter is very rudimentary and incomplete. Basic concepts like mean, median, and standard deviation are briefly discussed, but there is no mention of statistical random sampling techniques and confidence levels, which would surely be of significant importance when measuring key indicators across large populations, where a complete enumeration is either impossible, or too expensive and time-consuming. Sometimes, metrics which are "meaningful", are not the ones that are "tangible" and "easy to measure". A certain degree of statistical sophistication can be helpful in such situations.

In summary, the book offers some useful insight and practical advice for those who are charged with running an information security management program, but a healthy skepticism of the assumptions underlying the author's conclusions is warranted. In order to develop truly meaningful information security metrics, a much more sophisticated approach than what is described in this book will likely be needed.