First, here is what my early endorsement for the book said (can be found on the inside cover of the book):

“Amazingly useful (and fun to read!) book that does justice to this  somewhat esoteric subject - and this is coming from a long-time  visualization skeptic! What is most impressive that  this book is  actually 'hands-on-useful," not conceptual, with examples usable by  readers in their daily jobs. Chapter 8 on insiders is my favorite!”

What else do I think of the book, apart from the fact that it is awesome? :-)

First, I have to admit that I used to argue with Raffy about usefulness of visualization. I was burned by having to look at bad “visualization” tools and would take an ugly, meaningful table over an ugly, meaningless picture any day now. Thus, I was a visualization skeptic. Buy you know what? The book does justice to visualization really well, and it explains when to use it and when not to use it.

The book gives just the right amount of visualization theory, which is not onerous to read at all (unlike some other books), as well as other visualization basics. The fun starts at Chapter 4, where he covers  the process from data to useful pictures. This actually explains why some visualization are useful and some are not; if you just jam data into a graphing program, there is a good chance that it would not be too useful. If you follow the ideas from Ch4, it is more likely to be useful.

Ch5 and 6 cover network data analysis: logs, packets, flows. This is what most people usually try to visualize; this book goes beyond “worms and scans” into nice visuals of email traffic, wireless and even vulnerability data (I found the latter slightly confusing). Ch7 covers “compliance”, which, in this case, covers all sorts of fun things, from risk assessment to database log visualization.  As I said, Ch8 is my favorite: I agree that insider tracking MAY be the area where visualization tools and approaches beat others. In Ch9, the book covers a few visualization tools; obviously, including the author’s AfterGlow.

So, to summarize, get the book if you have any connection to security AND data analysis. In fact, it is very likely that if you are doing security, you’d have to do data analysis at some point and so will benefit from reading the book. And, yes, it does come with a CD full of visualization tools (DAVIX).

BTW, I am posting it at Amazon as well.

ABSTRACT

Googling for "SQL injection" gets about 4 million hits. The topic excites interest and superstitious fear. This whitepaper dymystifies the topic and explains a straightforward approach to writing database PL/SQL programs that provably guarantees their immunity to SQL injection.

Only when a PL/SQL subprogram executes SQL that it creates at run time is there a risk of SQL injection; and you'll see that it's easier than you might think to freeze the SQL at PL/SQL compile time. Then you'll understand that you need the rules which prevent the risk only for the rare scenarios that do require run-time-created SQL. It turns out that these rules are simple to state and easy to follow.

The report was written by a committee whose members include William Perry, a professor at Stanford University; Charles Vest, the former president of MIT; W. Earl Boebert, a retired senior scientist at Sandia National Laboratories; Cynthia Dwork of Microsoft Research; R. Gil Kerlikowske, Seattle's police chief; and Daryl Pregibon, a research scientist at Google.

They admit that far more Americans live their lives online, using everything from VoIP phones to Facebook to RFID tags in automobiles, than a decade ago, and the databases created by those activities are tempting targets for federal agencies. And they draw a distinction between subject-based data mining (starting with one individual and looking for connections) compared with pattern-based data mining (looking for anomalous activities that could show illegal activities).

But the authors conclude the type of data mining that government bureaucrats would like to do--perhaps inspired by watching too many episodes of the Fox series 24--can't work. "If it were possible to automatically find the digital tracks of terrorists and automatically monitor only the communications of terrorists, public policy choices in this domain would be much simpler. But it is not possible to do so."

A summary of the recommendations:

• U.S. government agencies should be required to follow a systematic process to evaluate the effectiveness, lawfulness, and consistency with U.S. values of every information-based program, whether classified or unclassified, for detecting and countering terrorists before it can be deployed, and periodically thereafter.

• Periodically after a program has been operationally deployed, and in particular before a program enters a new phase in its life cycle, policy makers should (carefully review) the program before allowing it to continue operations or to proceed to the next phase.

• To protect the privacy of innocent people, the research and development of any information-based counterterrorism program should be conducted with synthetic population data... At all stages of a phased deployment, data about individuals should be rigorously subjected to the full safeguards of the framework.

• Any information-based counterterrorism program of the U.S. government should be subjected to robust, independent oversight of the operations of that program, a part of which would entail a practice of using the same data mining technologies to "mine the miners and track the trackers."

• Counterterrorism programs should provide meaningful redress to any individuals inappropriately harmed by their operation.

• The U.S. government should periodically review the nation's laws, policies, and procedures that protect individuals' private information for relevance and effectiveness in light of changing technologies and circumstances. In particular, Congress should re-examine existing law to consider how privacy should be protected in the context of information-based programs (e.g., data mining) for counterterrorism.

Here are more news articles on the report. I explained why data mining wouldn't find terrorists back in 2005.

Developers have the objective of building a functional application. They are focused on building more functionality into applications. Moreover, building security creates more workload  for Developers which is a disincentive and moreover, Developers are rewarded for building more functionality than building more security. I have never seen a Developer in my professional life for being rewarded for building a secure application.

Hackers are focused on how to break the application. They look for weak links in application that will enable them to access application data. Developers usually follow process to build application, but Hackers have no process and all they have is multitude of possibilities. Hackers are innovative in trying various permutations in compromising the application.

A million dollar question is whether we can build secure applications when a Developer is focused on functionality but not on breaking the application?

There is a school of thought about Inside-out security where the application is built securely from scratch. Unfortunately,  this approach won't suffice because hackers traverse Outside-in. A little reflection will highlight the importance of vulnerability scanning and penetration testing of application. This will bring the perspective of what developers do not know already.

Building a secure application inside out is not enough. In order to address unknown unknowns (or blind spots of developers), penetration testing should be done. Both whitebox style penetration testing (where components of an application is known)  and also blackbox style penetration testing which mi micks an Hacker who may not have any knowledge of the application, should be carried out.

An application of higher level of security is not built just by Developers. It is built by integrative process of Developer mindset and Hacker mindset.  This is a constant struggle for years to come.
 

Originally, researchers in CEP were not trying to solve a problem of streaming data or streaming events.   Often we read this mischaracterization by folks in the database/streaming domain, as they were focused on the low latency processing of streaming events.   A natural extension of this research has been stream processing software (often called “engines”) that process streaming data with continuous queries, for example market data feeds for algo-trading or best market order execution.  This mischaracterization is partly responsible for why we see many order processing applications in market data stream processing mislabled as “complex event processing” applications.

The genesis of complex event processing was not the stream processing need for “feeds and speed” but the processing capability to solve what can be characterized as the “problem of asymmetric capabilties”.   The term “asymmetric” has been used in the military domain. For example we often hear the term “asymmetric warfare.”  However, in general the concept of “asymmetrical processing capablities” is the true genesis for CEP and related processing concepts and domains.   It is this genesis that distinguishes CEP from EDA, SOA, SOR, and so many other technology oriented concepts.

In order to illustrate what I mean by “asymmetrical processing capablities” we will take the example of the evolution of rocketry.    In the early days, scientists learned how to make rockets, I assume with gunpowder and similar chemical compounds to launch rockets.   Over many years the application of rocketry advanced much faster than the ability to understand the situations created in the sky.    In other words, folks could fill the skies with rockets long before they had the capability to track and identify (or sense and respond to)  the rockets in real time.

Therefore, the concept of “asymmetrical processing capablities” is the situation where there is a capability, such as “launch a rocket, sense-and-respond,” that is asymmetric in nature.    In other words, the capability to detect multiple rocket launches creates an asymmetric situation where it is easy to launch rockets, but hard to detect and defend against those launches.

The same concept can be applied to everyday air travel.   If we could only fly airplanes, but did not have the capability to track the planes, understand situations in airspace, and then respond to changing situations, air travel would be quite difficult.   Lucky for us, the global traveller, there is symmetry in the capabilities to build and fly aircraft and the capabilities to detect, track and follow the evolving situations in the sky.

The genesis of CEP was to solve the problem of asymmetry in cyberspace, or if you prefer, distributed data networks.   The folks who identified, early on,  the problems associated with asymmetry in cyberspace were folks working the the field of network and security management.    This is because there has been, and is currently, a great asymmetry between the capablities to “launch a process or transaction” in cyberspace and the capabilties to detect and track what is going on in the same domain.

In my next post on this topic, we will go into some details of this asymmetry and review the first CEP projects from Stanford University in the context of asymmetric processing capabilities in cyberspace.