Metricon 2 — Do Metrics Matter?

October 8, 2007

Metricon 2.0 was held August 7, 2007 in Boston.

Agenda #

• Keynote Debate: “Do Metrics Matter?”
  • Pro: Andrew Jaquith, Yankee Group
  • Con: Mike Rothman, SecurityIncite
  • Immoderator: Elizabeth A Nichols, PlexLogic
• Track 1. Chair: Gunnar Peterson, Arctec Group
  • Russell Cameron Thomas, Meritology – Security Meta Metrics–Measuring Agility, Learning, and Unintended Consequence
  • Fredrick DeQuan Lee and Brian Chess, Fortify – Security Metrics in Practice: Development of a Security Metric System to Rate Enterprise Software
  • Eric Dalci and Robert Hines, Cigital – A Software Security Risk Classification System
• Track 2. Chair: Jeremy Epstein, webMethods
  • Jeremiah Grossman, WhiteHat Security – Web Application Security Metrics
  • Brian Laing, Mike Lloyd, and Alain Mayer, Redseal Systems – Operational Security Risk Metrics: Definitions, Calculations, and Visualiztions
  • Anoop Singhal, NIST and Lingyu Wang, Sushil Jajodia, George Mason University – Metrics for Network Security Using Attack Graphs: A Position Paper
• Lunch – provided in the room
• Track 3. Chair: Adam Shostack
  • Chris Wysopal, Veracode – Software Security Weakness Scoring
  • Thomas Heyman, Christophe Huygens, and Wouter Joosen, K.U. Leuven – Developing secure applications with metrics in mind
  • Michael Gegick and Laurie Williams, North Carolina State University – Correlating Automated Static Analysis Alert Density to Reported Vulnerabilities in Sendmail
• Practitioner panel. Moderator: Becky Bace.
  • Three practitioners from thought leading companies describe how they use metrics to make better decisions. Slides
• Debate: Stump the Chumps
  • Security metricians spin the hamster wheel of pain
• Dinner – provided in the room

Dan Geer compiled a complete digest and a summary published by IEEE.

Program Committee #

Chair: Elizabeth Nichols, ClearPoint Metrics, Co-Chair

Members:

• Fred Cohen, Fred Cohen & Associates
• Jeremy Epstein, webMethods
• Dan Geer, Geer Risk Services
• Andrew Jaquith, Yankee Group
• Gunnar Peterson, Arctec Group, Co-Chair
• Russell Cameron Thomas, Meritology

Original Call for Papers #

Do you cringe at the subjectivity applied to security in every manner? If so, Metricon 2.0 may be your antidote to change security from an artistic “matter of opinion” into an objective, quantifiable science. The time for adjectives and adverbs has gone; the time for hard facts and data has come.

Metricon 2.0 is intended as a forum for lively, practical discussion in the area of security metrics. It is a forum for quantifiable approaches and results to problems afflicting information security today, with a bias towards practical, specific implementations. Topics and presentations will be selected for their potential to stimulate discussion in the Workshop.

Metricon 2.0 will be a one-day event, Tuesday, August 7, 2007, co-located with the 16th USENIX Security Symposium in Boston, MA, USA. Beginning first thing in the morning, with meals taken in the meeting room, and extending into the evening. Attendance will be by invitation and limited to 60 participants. All participants will be expected to “come with findings” and be willing to address the group in some fashion, formally or not. Preference given to the authors of position papers/presentations who have actual work in progress.

Each presenter will have 10-15 minutes to present his or her idea, followed by 15-20 minutes of discussion with the workshop participants. Panels and groups of related presentations may be proposed to present different approaches to selected topics, and will be steered by what sorts of proposals come in response to this Call.

Goals and Topics

The goal of the workshop is to stimulate discussion of and thinking about security metrics and to do so in ways that lead to realistic, early results of lasting value. Potential attendees are invited to submit position papers to be shared with all. Such position papers are expected to address security metrics in one of the following categories:

• Benchmarking
• Empirical Studies
• Metrics Definitions
• Financial Planning
• Security/Risk Modeling
• Tools, Technologies, Tips, and Tricks
• Visualization

Practical implementations, real world case studies, and detailed models will be preferred over broader models or general ideas.

How to Participate

Submit a short position paper or description of work done/ongoing. Your submission must be no longer than five (5) paragraphs or presentation slides. Author names and affiliations should appear first in/on the submission. Submissions may be in PDF, PowerPoint, HTML, or plaintext email and must be submitted to metricon@securitymetrics.org.

Presenters will be notified of acceptance by June 22, 2007 and expected to provide materials for distribution by July 22, 2007. All slides and position papers will be made available to participants at the workshop. No formal proceedings are intended. Plagiarism constitutes dishonesty. The organizers of this Workshop as well as USENIX prohibit these practices and will take appropriate action if dishonesty of this sort is found. Submission of recent, previously published work as well as simultaneous submissions to multiple venues is acceptable but please so indicate in your proposal.

Location

Metricon 2.0 will be co-located with the 16th USENIX Security Symposium (Security ’07).

Cost

$200 all-inclusive of meeting space, materials preparation, and meals for the day.

Important Dates

• Requests to participate: by May 11, 2007
• Notification of acceptance: by June 22, 2007
• Materials for distribution: by July 22, 2007