HANSEI - WHAT IS “RELENTLESS REFLECTION?” - And why we’re talking about it in the context of Risk Analysis.

Recall from yesterday’s post about how I got to thinking about the concept of Hansei-Kaizen, “relentless reflection” and “continuous improvement” and how we might apply that to risk management.  It’s a concept born of Toyota and is, in some way, the foundation for “Lean” production.

Call me biased, but I think that Hansei - the act of ‘relentless reflection’ made structured is the analytical function.  And I hate to debate (post-mortem) the father of Toyota quality success when he says that Hansei is the “check” in Plan/Do/Check/Act, but I think that Hansei also applies to the “Plan” of the P/D/C/A or Deming cycle.

You’ll recall the P/D/C/A cycle can be thought of even as an implementation of Scientific Method, in that it is Observation & Hypothesis Creation (P), Experiment (D), Analysis (Check), and Act (Revise/New Hypothesis, etc…).  Well then as such, the Hypothesis creation involves creating a model or creating an expected outcome for data using the currently accepted model.

So in our industry there is an opportunity for Relentless Reflection in both the Observation and Hypothesis (Plan) creation steps, and the Check step.  We create an estimate for control strength, or probable losses in the context of risk- then we go to Experiment step.  That hypothesis can be put it into production, have an audit, have a penetration test, whatever, in the context of the Do step.  BTW - using Hansei/Analytics in Plan is one way that strong analytical functions can really make penetration testing more useful - as a means to test the estimates and inputs into a model.  It’s Penetration Testing 2.0!  (<- tongue fully in cheek, yes)

Those who are versed in the reasons to merge Six Sigma and Lean together are probably already seeing where I’m going with this today.  But before you think that a simple DMAIC function is all that is needed to create proper “Hansei”, let me encourage you to keep reading.

Now if the analytical function can said to be “reflection”, why must it be relentless?

One word.  Change. There are essentially four separate “landscapes” or sources of change that we face (more on those tomorrow).  But anyone who has tried to manage system compliance, log management or policy exceptions knows that change is possibly the most difficult thing we security professionals must manage.  And when you think about it, there aren’t too many other business functions like information security where significant visibility and insight about the environment is needed for “complete” information (get bullish on Log Management is my recommendation).

HANSEI STEPS ADAPTED TO INFORMATION SECURITY

This is one of those quality control concepts that we can mangle adopt.  At Toyota, Hansei-Kaizen includes the following basic steps:

1. Initial problem perception
2. Clarify the problem
3. Locate area/point of cause
4. Investigate root cause (using an ask why 5 times approach)
5. Countermeasure
6. Evaluate
7. Standardize

Now it’s important to note that part of this includes the concept of Go See For Yourself, called “Gemba“.  Gemba can be translated as “the actual place” or “the place where virtue or truth is found.” At Toyota this might mean going to the shop floor to see the issue at hand in the production line.  But for us, that’s a problem because we live in the virtual world.  There’s usually not much use in hanging out in the wiring closets to try to see the problems.

But if you combine the concept of Gemba with the concept of “Nemawashi” –the process of discussing problems and potential solutions with all those affected- we can forge a similar concept using risk analysis.  That is discussing the issue and the risk associated with an issue (what some people would call “risk management”) with the business/LOB/data owner and let them accept authority and the risk decision.  We, the risk analyst, our goal is simply to perform items 1-5 (presenting countermeasure options that include transferring or accepting risk).  By going to the line of business and involving them, responsibility is shared.  Also, if you structure organizational behavior right, personal risk is transferred!

This sort of approach is also in harmony with concepts like “mutual ownership of problems,” or “genchi genbutsu,” (solving problems at the source instead of behind desks), and the “kaizen mind,” (an unending sense of crisis behind the company’s constant drive to improve).

One of the criticisms I have with the way most people try to implement DMAIC into “Lean”

REQUIREMENTS

Now to get this done, I really see three significant requirements.

1.)  A change in political structure.

2.)  Models that provide consistent, defensible analysis.

3.)  A Quantitative approach.  This means using actual units of measurement (not just amorphous percents, ordinal scales, etc.)  for risk and it’s subsequent factors.  Sure there are times when Q&D qualitative approaches are acceptable, but policy should be to have quantitative analysis whenever and wherever possible.

That last item - the quantitative approach - is really quite important.  And the reasons why will be discussed further in tomorrow’s post:

“What should we be reflecting about? & What is needed for reflection?”

P.S.  Your comments and suggestions, as always, are welcome.

P.P.S  Those who may be familiar with Lean/SixSigma/Kaizen sorts of mashups may be thinking - “hey, an Analytical step is built into SixSigma”.  Well, yes there is some prevision for analytical functions based on statistics, but I find SixSigma geared towards creating a State of Knowledge about operational processes, not towards creating a State of Wisdom for CISO’s around security & risks “big questions”.  In otherwords, the analytical function in DMAIC is in the context of Kaizen, and a different step than “reflective” analytics.

Learn how file integrity monitoring solutions work and the capabilities you should expect your solution to have. Then review a detailed checklist you should complete before purchasing your solution. Finally, discover how Tripwire Enterprise effectively combines file integrity monitoring with configuration assessment-a single configuration control solution that proactively assesses and monitors the IT infrastructure and enables organizations to achieve and maintain compliance with standards and regulations.

So, Anton Security Tip of the Day #16: Virtually Screwed - Journey Into VMWare ESX Log Analysis

CISecurty guide for VMWare (here) and DISA STIG for virtual machines (here) both mandate collection and analysis of VM platform logs; none goes into enough details on what to look for in logs. Let's try to shed some light on security-focused log analysis of VMWare ESX v. 3.x logs.

First, at least until ESXi becomes the default choice, one needs to keep in mind that ESX as "Linux-inside" and thus diving into /var/log will not reveal any "alien technology" (well, not much :-)). However, one of the most useful logs is /var/log/hostd.N which is not a descendant of Linux standard logs. Extensive VM event records are written into this file.

Let's focus on various types of logins to the ESX platform and identify logs that indicate a successful and failed attempts to log in. Here are a few useful examples to analyze:

Successful logins:

• May 30 09:20:42 esx2 su(pam_unix)[9405]: session opened for user root by jhonny(uid=1626)

This is a classic Linux root login message; you can watch for these by searching VMWare ESX logs for "session AND opened AND user AND root."  Notice the user name of the user who switched to root.

• May 30 09:20:34 esx2 sshd(pam_unix)[9364]: session opened for user jhonny by (uid=0)

This is also a classic Linux message for a normal (non-root) user login.

• [2008-05-25 06:57:48.774 'ha-eventmgr' 111639472 info] Event 40645 : User jhonny@1.1.1.1 logged in

This is a VMWare -specific application login to ESX. You can track such events by username, by event ID or by keywords "event AND logged AND user" (if you are using search)

Failed logins:

• May 30 09:20:31 esx2 sshd[9356]: Failed password for jhonny from 1.1.1.1 port 54773 ssh2

Another classic Linux message from the ESX system; a failure to login due to incorrect password.

• May 27 12:06:59 esx2 sshd[4756]: Failed password for illegal user jonny from 1.1.1.1 port 30594 ssh2

A message indicating a failure to login due to incorrect username (note a typo).

• May 25 07:03:48 esx1 sudo:     jhonny : 3 incorrect password attempts ; TTY=pts/0 ; PWD=/var/log ; USER=root ; COMMAND=/bin/bash

This ESX Linux platform message should also be familiar to Linux/Unix admins: it indicates multiple sudo password failures; look for such messages in the logs.

BTW, do you need to be reminded to track NOT only failed, but also successful login events?!

Overall, you must prepare for the future by learning to analyze  VMWare logs, just like you handled "legacy OS", such as Linux/Unix and Windows.

As I said before, I am tagging all the tips on my del.icio.us feed; here is the link: All Security Tips of the Day.

IBM announced today that it was spending US$300 million to build out 13 data centers in 10 countries in 2008 - IBM refers to these sites as "Business Resilience service delivery centers". These centers will certainly help IBM deliver more of its traditional IT recovery services but they will also support the next generation of IT continuity services - repeatable, scalable, productize services such as online backup and virtual recovery.  These types of services don't require massive capital investment in an inventory of heterogeneous server and storage platforms, instead the service provider can focus its efforts on building a scalable pool of virtualized servers and shared storage built with industry standard components.

Online backup is an important service because it provides an affordable information protection service for small and medium businesses and it's even useful for enterprises as a means to backup PCs corporate-wide as well as small servers at remote locations. In addition to the $300 million that IBM is spending on its new resiliency centers, late in 2008, it acquired Arsenal Digital Solutions, one of the major players in online backup.

In addition to online backup, recovery services using software-based replication to a cloud infrastructure will also open up new opportunities. These services will provide a much a better recovery time and recovery point than tape-based services but won't cost nearly as much as custom services based on storage-based replication and dedicated hardware. The cost of these services is more than most small and medium, even some large enterprises can or are willing to pay for. SunGard was the first to announce such a productized service, Forrester expects all the traditional DR service providers to bring similar offerings to market over time.

These cloud-based service offerings are important for several other reasons, first, it could help stem the tide of enterprises who are just so fed up with the traditional disaster recovery services model that they take DR back in house, second, it could convince, more medium size businesses that they can afford more advanced IT continuity solutions and lastly, it will help protect their market against new competitors who can simply partner with cloud providers such as Amazon S3 and Google to offer similar services.

IBM is not only using its expansion and acquisitions to stay competitive, it's also also hoping that customers will recognize the value of IBM expertise, process and best practices in BC.

What do you think? Does the reputation and expertise of BC and IT Continuity service providers like IBM and SunGard critical in your decision-making or can new players enter the market? Do these lower cost services that offer better RTO and RPO renew your interest in service providers or do you still plan to keep DR in-house?

I welcome your thoughts.

adaptive process management (n.) an element of resource and business process management, adaptive search and event processing. Sometimes referred to as “Level 4” event processing or process refinement.

application concept (n.) a definition of a set of properties that represent the data fields of an application entity. An application concept can describe relationships among themselves. For example, an order concept might have a parent/child relationship with an item concept. A department concept might be related to a purchase requisition concept based on the shared property, department_id. Application concepts can include an application state model.

application state modeler (n.) a UML-compliant application that allows you to model the life cycle of a concept instance — that is, for each instance of a given concept, you can define which states it will pass through and how it will transition from state to state. States have entry actions, exit actions, and conditions, providing precision control over the behavior of an event processing agent. Transitions between states also may have rules. Multiple types of states and transitions maximize the versatility and power of the application state modeler.

derived event (n.) an event that is created as a result of processing one or more other events.

complex event (n.) an event that is a situation-entity abstraction of two or more simple, derived or other complex events.

complex event processing (n.) CEP is a technology for extracting information from message-based systems. CEP is primarily an event processing concept that deals with the task of processing multiple events from an event cloud with the goal of identifying the meaningful events within the event cloud. CEP employs techniques such as detection of complex patterns of many events, event correlation and abstraction, event hierarchies, and relationships between events such as causality, membership, and timing, and event-driven processes.

event (n.) a instance of an event definition. It is an immutable object that represents a business activity that happened at a single point in time. Just as one cannot change the fact that a given activity occurred, one cannot change an event — events are immutable.

event aggregation (n.) the aggregation of simple, derived or complex events into higher levels of event abstractions.

event definition (n.) a set of properties related to a given activity that represents an important or interesting change of state in a human, system or computational activity. An event definition includes event properties such as event priority, event time to live (TTL), and a description of the payload, which is comprehensive information related to the activity that occurred. Events expire when the TTL has elapsed, unless the event processing agent has instructions to consume them prior to that time.

event channel (n.) a communications channel in which events are transmitted from event source to event receivers, typically received as electronic messages. Each channel can have multiple destination and. events can be configured to transmit to a default destination. JMS is an example of an event channel.

event cloud (n.) a partially ordered set of events (poset), either bounded or unbounded, where the partial orderings are imposed by the causal, timing and other relationships between the events. Typically an event cloud is created by the events produced by one or more distributed systems. An event cloud may contain many event types, event streams and event channels. The difference between a cloud and a stream is that there is no event relationship that totally orders the events in a cloud.

event-driven (n.) the behavior of a human, system or computational entity whose execution or actuation is in response to events, typically received as electronic messages.

event-driven architecture (n.) an architectural style for distributed computing applications in which some of the components are event-driven and communicate by means of events.

event processing (n.) computing that performs operations on events, including modifying, creating and destroying events.

event-object (n.) an software object that represents an event, generally for the purpose of computer processing, that exhibits both encapsulation, inheritance and polymorphism.

event prediction (n.) computational activity where the impact of events, complex events, and situations caused by events identified, including both opportunity or threat. Sometimes referred to as “Level 2” event processing, impact assessment or predictive analytics.

event pre-processing (n.) computational activity where events are cleansed or normalized to produce semantically understandable data. Sometimes referred to as “Level 0” event processing.

event processing (n.) computational activities on events dealing with the association, correlation, and combination of event data and information from single and multiple event sources to achieve refined identity and situation estimates for observed event objects, and to achieve complete and timely assessments of opportunities, threats, and their significance. Event processing is characterized by continuous refinements of event estimates and assessments and by evaluation of the need for additional sources, or modification of the process itself, to achieve improved results.

event processing agent (n.) an EPA is a computational entity that performs event processing.

event processing network (n.) a set of event processing agents and a set of event channels connecting them.

event properties (n.) data representation of an event, typically by name-value pairs of type string, integer, real, boolean or a complex data type.

event refinement (n.) filter, identify and track events & make initial processing decisions based on association, correlation and state estimation. Sometimes referred to as “Level 1” event, or event-object, track and trace.

event stream (n.) a time-ordered sequence of events. An event stream may be bounded by a certain time interval or other contextual dimension (content, space, source, certainty), or be open ended and unbounded.

event stream processing (n.) a time-ordered sequence of events. An event stream may be bounded by a certain time interval or other contextual dimension (content, space, source, certainty), or be open ended and unbounded.

rule (n.) defines what triggers unusual, suspicious, problematic, or advantageous activity within an event processing agent and what the EPA does when it discovers these types of activities. Rules execute actions based on certain conditions on events, instances, or a combination of both. A rule includes a group of condition-rule statements and action-rule statements. The condition statements instruct the EPA what to look for in events, and action statements instruct the EPA how to respond when conditions are met. If all the conditions in a rule are satisfied by events or instances or both, the EPA fires the actions. The action might be to execute tasks, create an event instance, modify property values in an event instance, create and send an event, or something else.

rules engine (n.) a type of event processing agent that uses a declarative programming model to process events. Formally described as “an abstract structure that describes a formal language precisely, i.e., a set of rules that mathematically delineates a (usually infinite) set of finite-length strings over a (usually finite) alphabet“. Informally, it can be any system that uses rules, in any form, that can be applied to data to produce outcomes.

rule language (n.) is an artificial language that is used to control the behavior of an event processing agent. Rules languages, like human languages, have syntactic and semantic rules to define meaning.

situation refinement (n.) identify situations, or complex events, based on event clustering, event-event relationships and relationship analysis and context. Sometimes referred to as “Level 2” event processing.

simple event (n.) an event that is not an abstraction or composition of other events.

virtual event (n.) an event that is imagined, modeled or simulated.