We read many stories on the net where folks claim that the current financial crisis could have been avoided with more or better use of technology.     This is expected, as software companies and IT professionals will often try to piggy-backtheir business development strategy on the “crisis of the day” to sell more goods and services.    Honestly, in this current situation, the main technology that we needed was simple, accurate financial models.

For example, in the chart above, the US economy was doing quite well with US federal funds rates low.   Housing prices in the US were skyrocketing and there was a concern about inflation.    There was an understandable concern the sustainability of that economy.

So, in perhaps one the most ill-advised Federal Reserve actions of many decades, the folks at the helm of the Fed decided to raise their lending rates around 500 percent over a two year period.

As we all know, primarily because of the action by the Fed, the world faces perhaps the worst economic disaster in modern times, while the US Executive Branch and the Congress fight over how to spend $700 Billion taxpayer dollars to inject liquidity into the markets to try to head off a global financial disaster.

It is amazing to me that the US Federal Government, or their advisors, does not have simple financial models with cause-and-effect analysis such as:

• Homeowners with adjustable rate mortuages will not be able to make payments;and
• Housing prices will fall dramatically; then
• Homeowners will default on loans where the collateral is much less than the asset value, and
• Banks will suffer great losses, and
• Lending will come to a halt, then
• Banks will collapse, then
• Wall Street will exit the markets in panic
• … and more trouble….. !!

There are and continue to be a lot of discussion and opinions about how risk management needs improvement. and I agree.   We will also read folks talk about how technology can be used to help solve this problem, including CEP/EP and related software (see also Capital Market CEP Fantasy Land). However, as much I would be pleased to see more CEP/EP applications and use cases, I do not believe that event processing technology is really very useful to solve the core problem of the current financial crisis.

The core problem is, seemingly, that our “financial experts” do not even have simple models that will illustrate what will or could happen when you raise the fed lending rates 500 percent in two years in an economy pregnant with adjustable rate mortgages.

To me, this does not appear to be rocket science.  The negligence by the US Federal Reserve and their advisors is astonishing.

For example, the “Average Joe Investor” does not care about “best order execution” or “smart order routing,” this is for “the big boys.”  As we all know, saving a few pennies or dollars per transaction to “Average Joe Investor” does nothing for them when their retirement nest egg is lost due to corporate greed and negligence.     The folks who “really care” about shaving a few milliseconds off market execution are the companies that are trading high volumes of exotic derivatives and baskets who have, for the most part, zero interest in the personal financial portfolio of “Jane in Iowa” or “Joe in Kansas.”

I am really amazed to see the dominance of greed in corporate America and the lack of corporate social responsibility.  Risk taking and “split second trading” does little for any small. individual investor and has proven to destabilize our society.    Who cares about saving a few pennies or dollars in market executive?

The answer: Only the greedy corporations, the same people responsible for the current destabilization, chao and near collaspe of our entire financial system.   Homes lost, unprecedented bankruptcies. and money market funds less than par value!   You no doubt have read that folks in the Reserve Money Market funds cannot even withdraw their “safe money.”  Investors in the Reserve Funds are being told that for every dollar they invested in a money market, they now only have 97 cents and cannot withdraw their capital as the Reserve waits for a government bailout.

What is to blame? Greed and profits over corporate social responsibility are to blame.

I read where some folks think the government needs to regulate market-related news, supposedly to stabilize trading based on news.   Regulating news has another name -  “censorship” - but who cares about the US Constitution when money and split second algo trading is involved?    I am amazed.   Folks in financial services just will say or do anything to make a buck, or keep from losing one, even at the expense of society and our basic constitutional freedoms.  News is not regulated in our democratic society, nor should it be to make algorithmic trading “better”.     What we need is less split second, computerized algo trading and more stablity.   Machine processing should not dicate nor mandate changes to our democratic principles.

Nor should our lives in a free society be censored or regulated because of the trading requirements for split second transactions that benefit large corporations.    The average investor does not need an unstable financial system trading exotic derivatives and baskets at the speed of light.  This requirement is driven by corporate greed that destabilizes the core economy and fabric of our society.

Of couse, many of the same folks would like for us to believe that technology is the answer.  This is a fallacy.

Corporate greed is destabilizing society.   What need to be regulated is not the news, but corporate risk taking and corporate goverance.  Individual investors do not need lightspeed transactions in an unstable world.   Citizens and families need a secure, stable economic infrastructure, something that has been lost in the culture of corporate greed, but hopefully not forever.

A similar arrogance is happening in CEP-land, where, it seems, each and every financial services event processing application is now a “CEP application” just because someone in capital markets puts “CEP” in the same paragraph. I find it ridiculous that the same market of folks who have helped destroy the global economy are now the world’s self-proclaimed authorities on complex event processing. Amazing, if you really think about it, isn’t it?

I read many posts these days by folks in the capital markets trading world, claiming their message routing application is “CEP,” or their algo trading application is “CEP,” - feeds and speed, typical of what “turns on” the financial services folks. As an editorial note: I recall when I worked for a software company, folks on the same team who worked on Wall Street would look down on folks with many years of IT experience outside of financial services. Some would say “he is only a security guy” in their attempt to put down anyone who does not have trading floor IT experience on their resume. I found it all quite ridiculous and foolish.

My resume, for what it is worth, has a number of financial services companies, including either assessing, architecting or building large scale security systems for S.W.I.F.T, Chase or SBC. This experience does not seem to “count” with the trading floor folks, since security is more about getting things right, not just supporting a form of gaming or gambling with other peoples money, with more feeds and speeds the better.

Of late, as I have watched the CEP/EP space evolve, and unfortunately, I see a similar type of “capital markets virus” spreading into CEP-land. Folks on the trading side of financial services seem to think that whatever they say or do is right, and whatever others outside of the trading side do is wrong. These folks are quick to ridicule others who have far more experience than they do, outside of the trading floor of capital markets.

After all, mostly what they do on the trading side is route orders - and if a little old lady in a small town in Iowa loses her life savings because of a bad investment decision, it means little to the folks on the trading floor, the market folks are into feeds and speed - just keep the beast alive. Place your bet on this market or that one! Away we go, faster and faster!!!!

I am sometimes a little sad to observe the same audacity in the CEP world. Instead of focusing on the hard complex problems that require accuracy, the original set of problems defined when the phrase “complex event processing” was minted, the capital market folks have hijacked the term for their marketing purposes in algo trading and order managment systems. These same people ridicule others who are working to solve the (originally stated) complex event processing problems, problems the capital market traders seemingly cannot understand, since they have never worked on complex network or security management problems.

Nevermind, that these “ultra low latency” systems cannot accurately detect a complex money laundering scheme or an elaborate fraud. Nevermind that these “CEP engines” cannot accuracy insure that Average Joe does not lose his hard earned money in a fraud scheme.

I have no problem with folks in capital markets using the term CEP, but they should not ridicule those in technical areas that are not focused on keeping the “trading beast” alive so people can lose their life savings in a blink of an eye; but instead focused on solving complex problems such as the class of problems called out when the three letter acronym “CEP” was created.

Blogger: Dan Blum

With the crisis in financial markets still unfolding, it is important to draw what lessons we can from the experience. Since the roots of the crisis lie in a monumental failure of risk management, it’s important to understand more about what happened, and then draw some parallels to our business risk management and  IT risk management situations.

The risk management failure in the housing market and on Wall Street had multiple interdependent dimensions:

• Mortgage lenders abandoned long standing prudent loan practices. They made too many loans that buyers might not be able to repay. Exotic instruments like ARMs, option ARMs, and interest only loans proliferated. In many cases, all pretense of lending standards were abandoned, so-called “liar loans” approved.
• Capital was grossly over-leveraged. Mortgage lenders and other financial services packaged loans into securities, which they sold to raise capital to support more lending. Real capital reserve requirements to back loans were reduced. Of course, if borrowers could not repay loans, all or parts of the derivative securities would become worthless.
• Risk was aggregated at Fannie Mae, Freddie Mac, and mortgage loan insurance companies. These companies bought or insured some mortgage loans, providing something of a backstop should loans fail. Government sponsored enterprises (GSEs) Fannie and Freddie in turn became over-leveraged and securities that they sold were in turn repackaged in the murky brew of mortgage-backed securities called collateralized debt obligations (CDOs) and other exotic instruments returning generous yields.
• Non-Caveat Emptor. Institutional wealth funds and financial services firms who should have known better bought securities that had been deliberately structured to obfuscate risk. They bought securities they didn’t understand with buried tranches of toxic subprime loans..

It was a great Ponzi scheme – one that kept working as long as housing prices were going up; the recipients of subprime loans could always flip that house to the next buyer. Everyone made money. As Chuck Prince of Citigroup famously put it during a July, 2007 interview: “So long as the music is playing, you’ve got to keep dancing. We’re still dancing.” But one month later, the music stopped. Since then, Citigroup and other financial institutions have taken massive writeoffs with more to come. Wall Street titans like Bear Sterns, Lehman Brothers, Merrill Lynch, and AIG have fallen or been bought out.

What can we learn from this risk management debacle?

As business risk managers and investors, we should ask questions like these:

• Does the executive incentive structure of the company encourage managers to dance around risk? Many Wall Street firms paid senior managers 5 times their salary in bonuses tied to annual growth alone.
• Is the company over-leveraged? Is it borrowing too much money and betting it on ventures with uncertain outcomes?
• Are financial models used for risk management realistic? Earlier, I described the mortgage market of the past few years as a Ponzi scheme, where risk management models must have assumed prices would keep rising. Unlike the dotcom boom whose demise many predicted, very few in the industry foresaw the sharp declines to come in housing prices and sales volumes. Historically, the U.S. housing market has been a steadily rising one, but on the other hand the 2000s saw unprecedented rates of price increases. In reality, what goes up must come down.
• Has your company’s risk council ever performed worst case scenario analysis and built adequate reserves? In the days before economics emerged as a would-be “hard” deterministic science, business leaders may have been more cautious, more aware of and more accepting of uncertainty. Events like the Great Tulip Bubble came once in decades or centuries – not every few years. Note that legendary investor George Soros has proposed a Theory of Reflexivity that, if true, helps explain the recent extremes of boom and bust cycles. This theory holds that market participants model market behaviors based on self-interest, and for a time, their manipulations change the reality of the market – until gravitational forces bring it back to earth. Has the music of ephemeral success played to the backbeat of deterministic-sounding economic models gone to your heads and infected your risk management models?
• Are cost cutting efforts pursued blindly? Outsourcing and other forays into treacherous global waters may be giving away the crown jewels. Smart companies cut costs, but they do it in smart ways. Smart companies think like intelligence agencies as they parcel out work to different partners with varying levels of dependability, and they check on those partners.

Risk management failures can also occur at the more technical level of IT security. As IT risk managers, we might ask questions like these:

• Are the accounting and financial systems your IT department supports under adequate control? As Fred Cohen wrote in one of our documents: “Many companies use computers to manage financial systems, and despite the Sarbanes-Oxley Act (SOX) claims about accounts being properly kept, there are many attacks on financial systems that remain. For example, most of the largest financial systems in the world running on common financial databases do not use double-entry bookkeeping and are thus susceptible to all manner of frauds by insiders.” We find it troubling that a prudent control dating back to the 12th century is going out of style in the name of convenience and cost cutting. Kind of like credit checking became anachronistic during the housing bubble, eh?
• Is the “separation” in your “separation of duty” (SoD) for real? Sure the SOX auditors are looking for SoD, and maybe you have different administrators with different accounts maintaining different systems or functions. But when they say Western civilization may be but one weak password from collapse they’re not lying. Look what happened to Sarah Palin’s email account! Weak and straggly SoD is a problem across all critical IT systems where deperimiterization and server consolidation may be bringing down protective barriers, identity management is weak, and strong process controls (e.g., where two people must sign on, one perform a critical operation such as backbone router reconfiguration, and the second observe) abandoned in the name of expediency.
• Are risks being aggregated to unacceptable levels in centralized control systems? There are many ways that risks aggregate within enterprise IT infrastructures as we pursue automation and cost cutting. Network risks aggregate when centralized domain name system control is implemented. Application risks aggregate when common infrastructure is shared among applications. And enterprises aggregate platform risks when they use low-assurance endpoints, authentication, and directory systems with single sign-on to access large numbers of resources and don’t separate high consequence systems.
• Non-caveat emptor: Has IT security really done the worst case consequence analysis, attack graphs, and vulnerability analysis to know when putting more eggs in a supposedly stronger basket aggregates risks to an unacceptable level? Or are you depending only on vendor claims about some black box appliance equivalent of a risk-obfuscated CDO security? Caveat emptor (buyer beware) again! (The good news is we’ll keep talking about promoting vendor and product rating systems so you don’t have to do all the detailed product analysis yourself, but that’s another post.)

There are many parallels between the monumental risk management failure in the financial markets, and the probable weaknesses in our day to day business risk management and IT risk management. Abandonment of prudent practices for profit; excessive leverage and centralization; ill-constructed risk analysis models; risk obfuscation; and a failure of caveat emptor seem to be common problems. Please take this as a wakeup call to sharpen up the risk management thinking, process, and execution.

In the case of phishing, it is relatively clear. The developers believe the PKI book. The PKI people believe in the efficacy of digital signatures to prove stuff. The cryptographers believe in the perfection of mathematics, and the security world believes in the completeness of their own learning. They are all wrong, but only at the large level of generalisations, not at the detailed level of particular claims. Any one of the claims, in isolation can be shown to be true. But, generalising these brittle claims to be solid building blocks is a completely different question. Few of the claims are strong enough to partake in a general model without severe support; the general model of secure browsing is the best evidence of how it is secure in name only.

How then is it built? By accident or by design, a series of claims meet together in a holy ring of righteous architecture. Each of the proponents claim loudly that their part is strong, but the ring has no strength. Eventually, one of the claims in the links is broken. For phishing, the browsers never did have the potential to show authenticity; not only did they not have the security strength to do it (c.f., Skype v. CSRF), they didn't even do it in practice (recall the lost padlock?), and their recent efforts to show authenticity (c.f. colour debate) reveal how far they are from understanding even the goal, let alone the implementation. Once that link was broken, and money was made, all the others revealed their weaknesses, as crooks systematically worked to breach the lot.

If we look at the wider financial collapse, now underscored by the nationalisation of the worlds biggest financiers of mortgages ($ 5.3 trillion.... or is it $ 5.4 ?), we see the same pattern. The bankers believed in their product. The originators believed in their origination, the securitizers believed in their free market and accurate price, and the holders believed in the assets. The CDO, the subprime, the other 100 special names, each was a contract. Each was clear in and of itself. But, when placed end-to-end, in a line, with a bunch of other agreements, the claims that were good in isolation were not strong enough to participate in the super-claim made of the overall edifice.
The financial system was built like a bridge; each piece rested on the previous one. And then, the clever architects bent the bridge around ... and around again, until the first piece met the last. The elegant keystone of finance was to finally lift up the first one to rest on the last.

Thus, the banks themselves invested their capital in their own product.

I agree with Mark that SOR is important and very interesting; but in his reply he seems to be confusing CEP with “complex EAI” or a “complex messaging” application.  For example, Mark says,

“It’s not uncommon for a single SOR system to connect to 10 or more markets and multiple asset classes.  Not only is this a confluence of events, it’s a stunningly complicated environment in which to create a complex, real-time model in which to apply “simple” routing decisions. On this basis alone, SOR needs CEP.”

Connecting to many market feeds with multiple asset classes might be complicated, but “complicated connections” are an EAI  (adaptation layer) function, not a core CEP function.   In fact, TIBCO Software has been doing this type of low latency back-office order routing for many years, and TIBCO historically calls this “messaging.”  Adding some rules to high speed, low latency messaging does not make it a “CEP” application.

Mark goes on to set up a counter argument to ILOG’s Changhai Ke, comments with,

“SOR operates by analyzing the confluence of events from market data feeds, order flows from OMS systems, and executions, aggregating and analyzing those events in real time, and adjust routing decisions on the fly.”

This is the well travelled argument the “new stream processing vendors in capital markets” have been saying, still unconvincingly, for the last few years.  Basically their perspective is that if you have a lot of ”feeds” and a core requirement for “speed” - “feeds and speed” - you are doing “complex event processing.” 

Mark Palmer forcefully stated his opinon that the folks who do not agree with him do not “understand” modern day SOR.    However,  a strong counter argument can be made that the “newcomers” to capital markets like StreamBase do not understand that “feeds and speeds” with order routing is little more than moderan day EAI.   This is a basic message routing capability and it has been around for a long time.  After all, Wall Street operated quite well before the term CEP was coined!  TIBCO technology was providing Wall Street back office, low latency, smart order routing a decade ago, and they called this technology “messaging”.  

So, I remain unconvinced, at least by Mark’s passionate counter post, that SOR is CEP.   SOR, as Mark and other have described it, is a low latency messaging technology.  Message routing rules have exisited in this technology space for decades.

I agree with Mark completely that low latency EAI (like SOR has been described) can be quite complex, from a “feeds and speeds” perspective.   However,  I remain skeptical that “feeds and speeds” is much more than  modern day messaging and message routing.

In closing, in the network and security management world we have been dealing with “myriad feeds and speeds” for as long as I can remember, but admitted not like capital markets.    Taking myriad feeds, running rules against the feeds and then routing the messages/events for further processing, regardless of the complexity of the feeds and the data, is actually more of a messaging/ESB technology than a CEP technology. 

I remain completely open minded to any convincing counter arguments.

IBM announced today that it was spending US$300 million to build out 13 data centers in 10 countries in 2008 - IBM refers to these sites as "Business Resilience service delivery centers". These centers will certainly help IBM deliver more of its traditional IT recovery services but they will also support the next generation of IT continuity services - repeatable, scalable, productize services such as online backup and virtual recovery.  These types of services don't require massive capital investment in an inventory of heterogeneous server and storage platforms, instead the service provider can focus its efforts on building a scalable pool of virtualized servers and shared storage built with industry standard components.

Online backup is an important service because it provides an affordable information protection service for small and medium businesses and it's even useful for enterprises as a means to backup PCs corporate-wide as well as small servers at remote locations. In addition to the $300 million that IBM is spending on its new resiliency centers, late in 2008, it acquired Arsenal Digital Solutions, one of the major players in online backup.

In addition to online backup, recovery services using software-based replication to a cloud infrastructure will also open up new opportunities. These services will provide a much a better recovery time and recovery point than tape-based services but won't cost nearly as much as custom services based on storage-based replication and dedicated hardware. The cost of these services is more than most small and medium, even some large enterprises can or are willing to pay for. SunGard was the first to announce such a productized service, Forrester expects all the traditional DR service providers to bring similar offerings to market over time.

These cloud-based service offerings are important for several other reasons, first, it could help stem the tide of enterprises who are just so fed up with the traditional disaster recovery services model that they take DR back in house, second, it could convince, more medium size businesses that they can afford more advanced IT continuity solutions and lastly, it will help protect their market against new competitors who can simply partner with cloud providers such as Amazon S3 and Google to offer similar services.

IBM is not only using its expansion and acquisitions to stay competitive, it's also also hoping that customers will recognize the value of IBM expertise, process and best practices in BC.

What do you think? Does the reputation and expertise of BC and IT Continuity service providers like IBM and SunGard critical in your decision-making or can new players enter the market? Do these lower cost services that offer better RTO and RPO renew your interest in service providers or do you still plan to keep DR in-house?

I welcome your thoughts.

Corruption, intimidation, robbery, violent assault, forgery, large-scale fraud. No, not the subject of the latest John Grisham novel, but sensational allegations, made public Apr. 4 by Hermitage Capital Management -- until recently the largest foreign portfolio investor in Russia. In a detailed and damning report, titled Criminal Justice -- Russian-Style, Hermitage alleges the fund's Russian subsidiaries have fallen victim to an elaborate con designed to defraud the fund of hundreds of millions of dollars. 
  
The most sensational part of Hermitage's allegations is that the attempted larceny was carried out with the direct connivance of officials in the Russian police. Hermitage alleges the police seized documents and equipment that were instrumental to the attempted fraud, which involved bogus court cases based on forged documents, the aim of which was to sue Hermitage subsidiaries for hundreds of millions of dollars. "The most shocking thing is not that there are corporate raiders in Russia who attempt to steal your shares," says Jamison Firestone, managing partner of Firestone Duncan, Hermitage's law firm. "The shocking thing is that the police worked hand-in-hand with them, and actually performed the theft of the documents so that the corporate raiders could then do their work."

So the two-pronged scam worked in one area and failed in another. The perpetrators weren’t able to steal the assets from us based on the fake court claims, but they were able to steal $230 million from the Russian government by filing amended tax returns on behalf of our stolen companies. What makes this story even more shocking is that we filed six 255-page criminal complaints with the Russian authorities in December last year, one month before the tax fraud took place, and they did nothing to stop it. Two complaints were sent to the Russian General Prosecutor, two to the Russian State Investigative Committee and two to the Internal Affairs Department of the Interior Ministry. There was enough information to prevent the fraud and indict a number of people behind it if the government had acted. 

Instead of doing anything to save the Russian state from this highly sophisticated and organized looting, two of our complaints were thrown out immediately; two were returned to the same Interior Ministry official we were complaining about (essentially, he was being asked to “investigate himself”); and one was thrown out for “lack of any crime committed.” Only one complaint was taken seriously. It was taken up by the Russian State Investigative Committee in early February, but before it could get any traction, the case was lowered to the South region of the Moscow district of the State Investigative Committee (the lowest level of the Committee) and by June, another senior Interior Ministry official whom we had named in our complaint had joined the “investigation” team (again, to “investigate himself”). To this day there has been no serious response by the Russian authorities to this massive fraud against the Russian state. 

As we described in our April letter, the problem of corporate “raiding” is now so endemic in Russia that President Medvedev speaks about it as one of the biggest problems faced by Russian businesses. In this case, raiders have taken this problem to a new and absurd extreme by “raiding” the Russian state itself and so far getting away with it. Together with HSBC, we will shortly be filing new criminal complaints with the Russian General Prosecutor and Russian State Investigative Committee as well as with many law enforcement authorities outside of Russia. It is hard to predict what will happen next in this unfolding and unbelievable saga, but as always we will keep you updated on any further developments as they arise.

Of course we see individual identity theft on a regular basis (actually as Ross Anderson points out its not really identity theft but poor controls on the bank's parts using SSNs as secrets and so on), but you dont see a major corporation stolen every day.

1. Orbitz by Streambase (Real-time monitoring)
2. Netbank by Coral8  (Algo trading)
3. LiquidNet by Coral8 (Market monitoring and analysis)

Progress Apama had a few algo trading references, but they (wisely in my opinion) are not (seeming) calling algo trading platforms sales, CEP, in 2008.   This is good (and more accurate) and I applaud Apama for building a great event stream processing platform and not overhyping the phrase “complex event processing” every opportunity they get.    Maybe we should create another award category?    

You will not find any “secret council of elders” here, nor will you find any subjective opinions about the market from people we send out email asking for their opinions - only the facts in an open transparent way.  Here is the Google worksheet, if interested.

Please contact me or comment here if we missed anything and we will take a look and we will add your suggestion if it meets the criteria.

Revisions:

1. Added LiquidNet/Coral8 suggested by Marc Adler (see comments)