Rating CIO’s–Who are the good ones? (part 1 of 2)


There are 4 classes of CIO’s.

CIO’s that live off Gartner reports are good CIO’s 27% of the time 3 times out of 10. Their strategies read like an IT buzzword generator in an infinite loop and technology illiterate CEO’s oft confuse it for intelligence or insight…. but only for a year or two.

CIO’s that are really just re-titled CFO’s who are really just focused on cost control are my second class of CIO.  Their IT group to them is just a drain on profitability and the objective is to have the least amount of IT at the least cost.  They also generally last a year or two after their Cost efficacy programs show their results in lower productivity, lower sales, lower profit and angry customers/employees.

The third class of CIO is the promoted IT Manager. This is the person who does understand the technology, knows what server gives the best uptime, knows how long a new system takes to develop and knows how much efficient IT operations should cost. They know all about ITIL and Service Level Agreements and they are rarely “taken” by a vendor on vapourware. There is also a very good chance they don’t know much about the business they run the systems for.  These people last until a smart outsourcer convinces the CEO that they know the business better than their internal people do, then the CIO and his/her management team is toast.

The fourth class of CIO is the business focused CIO.  They know enough about technology to place good people around them that can help them navigate technical waters safely.  Their focus though is on business. What the business needs to achieve to be successful and how IT can help make that happen is their most important objective. They manage a portfolio of initiatives and balance it to create both forward pushing innovation and introspective projects that help the company do things they already do, but do it better and more cost effectively. These are the CIO’s that become iconic in their organizations. These are the CIO’s that are always included in the board meetings. These are the CIO’s that provide true business value and to no surprise rate higher.

When a company writes-off a major IT initiative it almost always occurs with a CIO who is a member of one of the first 3 classes.

Buzzword Bobby wants to do ECM, EAI, CRM, SOA, BI, EDW, ESB and the Cloud without any idea of why or when or how to align it with the business. Their programs fail eight out of ten times or more.

Fiscal Freddy takes on outsourcing, consolidation, rationalization and deprivation projects that neuter the organization’s core capabilities. Their programs also fail eight out of ten times or more.

Techie Tina was a great Cobol programmer, PMO lead and IT department manager. She now brings all of that knowledge to bear on creating robust, efficient systems that ensure the organization makes no significant progress. The projects rarely fail to be delivered, but they still fail 8 out of 10 times to deliver enough business value.

In the fourth class I will provide the example of one of the finest CIO’s I have ever had the pleasure to work with. He was a banker.  For 25 years he had from starting in a branch giving loans, worked up through risk analysis, finance, treasury, marketing, GM of a group and then President of  the bank’s largest, most profitable division. He liked technology. He used it, he read constantly and he surrounded himself with a team of experts he could really trust. He took the CIO job because he understood how critical the role was to the future of the bank.  He did not read the business cases put forward for new major initiatives, he created them. He visited with each of the business groups asked specific questions on their strategy and got full alignment with them on what initiatives would deliver the benefits. When it came to the final business justification for a $100 Million initiative, it was the business stakeholders who asked and I quote:

“Are you sure that’s enough money? We really want to make sure this is properly funded.”

I would argue today that the most effective CIO is not the one that attends all the Gartner conferences, not the camouflaged CFO and not the really good IT person. The CIO that will rate the best is the business person who understands the intricacies and strategies of the organization they serve and has sufficient technical knowledge to know when to ask good questions and whom to ask them to.

Posted in Uncategorized | Leave a comment

Hiring an IT consultant–The importance of a contract


It does not have to be a 500 page legal contract, signed under seal and notarized but both parties in a consultancy agreement need a contract. Without a contract it is like playing solitaire with the four kings missing. You may play the game all the way to the end, but you will never win. 

Amongst the most important elements of the contract are:

  • Expectations of services and deliverables (“the what”) and the specific criteria that makes them acceptable
  • The definition of processes, methods, communication, standards and people (“the how”)
  • Expectations of what the client must do  or not do to enable your success
  • What you both will do if things change (“the what if”)

You can write it on a napkin, put it in an email or spray paint it on a conference room wall, but it needs to be written. It needs to be written not to enforce payment of the fee but for the most important purpose; to ensure the consultant and the client both agree on what and how something is to be done from the start and what defines “done”.

The worst possible consulting contract  for a consultant or their client  is to have a consultant parachuted into an engagement with a previously negotiated agreement; an agreement where the services are fuzzy, the deliverables are fuzzy, the expectations are crystal clear to everyone, and everyone has a different “crystal clear” vision.

If you are hiring a consultant into an existing engagement or you are a consultant coming onto one, take the time to review and re-align on the contact. It will make sure that you are playing with a full deck and  both parties can win.


Posted in Consulting Excellence | Tagged , | Leave a comment

Ethics in IT Consultancy – Primum non nocere


If you are a Medical Doctor, medical ethics are taught to you, you pledge an oath to uphold them  and the College of Physicians and Surgeons is there to remind you if your memory lapses with a  peer-controlled disciplinary action. At its very core, Medical ethics starts with “Primum non nocere”. First, Do no harm.

In professional consultancy we are often compensated as well as or more than our medical colleagues and quite often held to a much lower standard. We don’t of course have the responsibility of life and death … or do we?

I have worked for defence companies such as the former Paramax and Loral. Their “fire control” systems had nothing whatsoever to extinguishing a blaze and everything to do with extinguishing a life. I have worked on Medicaid systems that invoked prospective utilization rules to deny requested medical treatments and I have worked on ICU systems that have hastened someone’s trip to the morgue by reallocating ICU capacity. I have worked on welfare systems that knowingly exposed names and addresses of people in the Witness Protection program to people that should not have access. I have worked on an emergency management simulator that was buried very, very deeply because it showed real mortality projections of a accidental chemical leak. I have worked on an AFIS system where the implementation literally caused people to lose digits (anatomical). I have denied people the ability to vote, provided the ability for organizations to track someone’s movement and pushed people into bankruptcy. I have been well paid for every one of these.

As pay-by-the-minute IT contractors people simply implement what their clients ask for and ethics is never considered.  As professional consultants though our responsibilities should at a minimum include the requirement to inform our client of the potential impact of their decisions and yes in some cases to refuse to provide the service.

Let’s see if an IT version of the Hippocratic Oath works.

I swear to fulfill, to the best of my ability and judgment, this covenant:

I will respect the hard-won scientific gains of those physicians in whose steps I walk, and gladly share such knowledge as is mine with those who are to follow.

  • ( I will make use of existing IP and not reinvent the wheel. I will create IP for the use of others so that as a profession , we all get better)

I will apply, for the benefit of the sick, all measures [that] are required, avoiding those twin traps of overtreatment and therapeutic nihilism.

  • ( I will use every tool at my disposal to build the best solution. I will not build solutions that are over-engineered just to over-bill and  I will not engage in endless meetings that accomplish nothing)

I will remember that there is art to medicine as well as science, and that warmth, sympathy, and understanding may outweigh the surgeon’s knife or the chemist’s drug.

  • ( I will not treat my clients as customers. They are better than that.)

I will not be ashamed to say “I know not”, nor will I fail to call in my colleagues when the skills of another are needed for a patient’s recovery.

  • ( I will not be ashamed to say “WTF? I don’t get it”, nor will I fail to call in my colleagues when the skills of another are needed for a client’s project)

I will respect the privacy of my patients, for their problems are not disclosed to me that the world may know. Most especially must I tread with care in matters of life and death. If it is given to me to save a life, all thanks. But it may also be within my power to take a life; this awesome responsibility must be faced with great humbleness and awareness of my own frailty. Above all, I must not play at God.

  • ( I will respect the privacy of my clients. I will not play at God and must not act like Him either and will not engage in projects that put me in that position )

I will remember that I do not treat a fever chart, a cancerous growth, but a sick human being, whose illness may affect the person’s family and economic stability. My responsibility includes these related problems, if I am to care adequately for the sick.

  • ( I will remember that high risk projects include high personal risk for my client and will engage to mitigate their risk )

I will prevent disease whenever I can, for prevention is preferable to cure.

  • ( I will deliver a solution that is supportable )

I will remember that I remain a member of society with special obligations to all my fellow human beings, those sound of mind and body as well as the infirm.

  • ( I will try to be nice even to my crazy clients)

If I do not violate this oath, may I enjoy life and art, be respected while I live and remembered with affection thereafter. May I always act so as to preserve the finest traditions of my calling and may I long experience the joy of healing those who seek my help.

  • (If I keep the oath, I will have a long and enjoyable career as a professional IT consultant)

Not every client engagement has a significant ethical challenges, but some do. As a profession it is perhaps time to ask if we should; first, do no harm.

Yesterday, a motorist in California ran over and killed a 4 year old girl. The driver was filling out a CRM form on his iPad when he struck her.

Posted in Consulting Excellence | Tagged | Leave a comment

CIO – How to Fix your IT Group


Think of your current portfolio of IT work as the Colorado River for a moment. Some projects are very close, others further upstream. Important “Operational” projects are at the surface as they are required to sustain current operations and other pressing projects are also close but are beneath the surface.  These projects flow to and through your IT group and all IT groups have more work than is ever possible to process. Your IT group becomes the Hoover Dam.

The operational projects at the top of the dam are the first to flow through, spin through the turbines and generate the electricity to “keep the lights on”. Still pressing at the dam are the important other projects; the ones that provide innovation, differentiation and competitive advantage. The ones that are game changers for the organization. They too are right at the face of the dam, however they are at the base of it. The pressure at the base of the Hoover Dam is 45,000 pounds per square foot.  Moving that project to the top so it can get into the chutes and processed either requires herculean CIO effort or you have to fix the way your IT group works.

As the CIO you can

  • install more efficient turbines (new processes, training) or
  • upgrade your generators (new software, application development software and hardware, rationalize/standardize/optimize)

but the flow of water coming into the top of the chutes is always the same. If you have been in IT for a while you have lived through 4GL’s, Object Oriented Programming, SOA, AGILE etc. and still the backlog of projects remains. All of these technologies have promised paradigm-shift improvements in throughput and they have all delivered benefit, but all well below expectations.  Twenty years ago a mainframe Cobol development IT group could produce new systems at a productivity rate of 16 hours per function point. Today the average rate is still the same, with only a small percentage of IT groups in the higher 10-12 hour per function point range. Clients who off-shore development often find that the productivity rate is half of on-shore, so even with more, less expensive resources, the backlog remains.  So what’s the real answer?

Fix your IT Group.


The Hoover dam has 6 diversion tunnels at the base of the river.  Open these tunnels just a bit and the water comes flying out at 200,000 CFS and  120 MPH at 300 PSI. The interesting part of this analogy is that the diversion gates can be 75% open before they effect the net generating capability of the dam itself.

The concept here to fix your IT group is to find a completely different path to get the work done. Your IT systems and processes are not likely bad, but they are built to the highest common denominator. You use hardened datacenters, twin DR sites, dozens of security mechanisms, rigorous requirements analysis and testing and your best people are consumed by maintenance and compliance to the processes. All of these things are required, but they are not required for every system.  The same procurement process used to acquire more Tier-1 SAN space will be used for any project,  not just the one with the Petabyte requirements.

What if?

  • You don’t procure hardware?
  • You don’t have your own datacenter?
  • You empower your architects & developers to make light-weight infrastructure decisions?
  • You empower your team members to do multiple roles?
  • You give your IT personnel a credit card with a $5,000 limit instead of a Procurement Request form.
  • You don’t maintain exclusivity of your corporate directory?
  • You keep an open-mind to right-sizing requirements management, change management, configuration management and yes even quality assurance and testing processes?
  • You believe that security works well-enough to not represent a material business risk to your company?

Today with federated identities and cloud PaaS capabilities, your end-users would have no idea a new system produced for them was off-premise and running in a PaaS cloud. When integrating to your systems whether they are local or cloud based, the technology today does not differentiate and creates a transparency between them. I am not suggesting a wholesale migration to the cloud, what I am suggesting is that to fix your IT group you need to divert some resources away from the core flow and set them up in a completely different world.

Let’s assume for the moment you require a new application where your customers can check the status of an order, update a quantity, check and change a delivery schedule and securely approve a revised PO amount on the their mobile devices. (iPhone, iPad, WP, Android, Win, Mac etc.) For your 200,000 global customers you also want to create and publish interactive training videos to show them how to take advantage of this new capability and you also have an internal application for dealing with exceptions, purchase order limits and credit issues and call center integration for your customers if they run into issues.

Go ahead ask your IT group how long this will take…

My guess is they will tell you “6 to 8 months once we get started but at the moment we have a backlog of operational changes that won’t be cleared until mid next year”.

The real answer was this project from concept to operation was 8 weeks until the first production customer used the system. An additional 8 weeks later it was opened to all customers globally and was at least once used by 25% if their customer base within the first 6 months.  What do the customers say?  “Very Convenient, you’re making it much easier to do more business with you”

This was not a simple application, it had commercial grade security requirements, significant integration with  ERP systems, Finance Systems and Call Center systems, a requirement for a global content repository and required support across multiple mobile devices.

The CIO in this case needed this project done, done right and done fast.  She diverted a good team from the core  IT group and empowered them with the ability to build this on Azure PaaS.  The development environment was up day 2 (Tues) of the process, by Friday of that week they had concept screens and flows built for the following Monday workshop with the stakeholders.  While the work on the end-user application continued, team members built and tested connections to the backend datacenter systems with an on-premise integration server and 3 weeks later they had a dedicated private link to the cloud installed. By week four, they had a fully functioning solution  and the next four weeks were spent testing, creating  and publishing training materials and security hardening the solution.

As the CIO you cannot divert all of the processing away from the dam, you do need to keep the lights on. Divert some and you may just find out that you can divert more than you think. Set expectations very, very high for the diversion project, find some IT resources who share your vision, hire an experienced PaaS architect to be a coach and catalyst for your team and divert to the cloud for that next new project idea.

That’s how you fix your IT Group.

Posted in Technology | Tagged , , , , , , , | Leave a comment

The perfect consultant – The Ambivert?


Most of us are not pure introverts or extroverts. We exist somewhere on the continuum in between the two polar extremes.  The highest paid consultant in the world today is  Alan Greenspan who charges $120,000 per day for his time. He’s a real extroverted party animal. You can check Bing or Google and find thousands of pictures of his public rowdiness and table-dancing exploits. Or can you?

The vast majority of skills required to be a successful consultant are learned skills.  We can easily learn how to:

Communicate Bear with me for a moment…as we look at advice
  Giving Bad News and Feedback to your client
Facilitate Facilitating the “Angry Mob” meeting Part 2
  Facilitating the “Angry Mob” meeting Part 3
  Facilitating the “Angry Mob” Meeting
Influence Ambidextrous Influence
  Mastering the Art of Influence
Meeting Skills The People You Meet – Consulting Lessons from Goldie Hawn
Negotiate Negotiating the Curve (part 2)
  Negotiating the curve
Present Nailing the Presentation–10 simple rules
Problem Solve Problem Solving … The PHD course (part 2) Understanding the Problem Space
  Problem Solving … The PHD Course (part 3) Defining Success
  Problem Solving …. The PHD course (part 1) Permission
  When you’re right and the question isn’t…
Question A user will tell you anything you ask, but nothing more.
  The Great Question(s)…
Think Strategically Have you ever met Bill Gates?
  Thinking outside the box …

but it is MUCH harder to change who we are.

Ask yourself these questions and see which one of the 5 consultants ( A through E ) you resemble most.







I am a catalyst and  instigator and it draws people in.






I am successful when people focus on me, believe me and I am prominent.






I am skilled in handling people dynamics and interactions.






I like to be involved in maximum activity. Bustle is good, frenzy even better.






I make new friends with clients easily.






I am reserved around new people.






I don’t like to be the centre of attention or focus.






I don’t like to socialize outside work with my clients. Client lunches are full-on formal occasions for me.






I  work and think best on my own.






I do not require client interaction to be fulfilled in my role.













100% Extravert

70% Extravert

50% Extravert
50% Introvert

70% Introvert

100% Introvert

The desired characteristics of highly successful consultants are in blue. The Ambivert is the best all-round consultant with the ability to attain/retain focus from the client when required but still retains the capability to think creatively on their own without constant group input. If you scored high on the extravert scale we also have a role for you. It’s called sales.

Posted in Consulting Excellence | Tagged , , , | Leave a comment

Consulting companies that “GET” the value message and those that don’t.


Bain a global management consulting company employs 5,000 people, generates $2B in consulting revenue each year and “gets” value. Their web page makes the audacious claim that Bain clients outperform the market by a 4 to 1 margin. Do you think CEO’s pay attention to that?

They did not say “we’ll make you more productive” or “we’ll save you money” or “you’ll be better at managing client relationships”, did they?  They spoke directly to the CEO and said it’s about shareholder value and return, and engage us because our clients outperform dramatically.

To do it of course, the company will need to be more productive, they will need to lower costs, they will need to to be fantastic at sales and marketing among many other business disciplines.  IT consulting can drive real business value for our clients.

In IT there are  4 main directions to drive value.

  • Competitive Advantage – Develop that “thing” that peers do not have, drive customers to the business and retain them.
  • Better Management of the Portfolio – The ability to reduce the investment in status quo operations and free-up dollars for high value innovation that make the business better.
  • Execution – The ability to lower the cost of development, lower the time to market for new solutions and lower the cost and incidence of failure.
  • Operations – Systems that companies need, need to be usable, available, perform, be trustworthy, secure and private where required.

Competitive Advantage

Why is FedEx successful? Good Marketing? Good Delivery? Friendly drivers? or was the fact that they were the first courier company that could actually tell you where your package was at any time and the moment it was delivered?

The single largest value driver for a commercial client is competitive advantage. Competitive advantage drives revenue, profit, market share and shareholder  value. As consultants our largest contribution is helping out commercial clients find and execute on the solution that places them not just ahead but well ahead of their competitors and sustains them in the lead.

  • Fedex
  • Amazon
  • iTunes

Three companies were their IT competitive advantage is clearly the reason for their success.

Great IT Portfolio Management

The ability to reduce the investment in status quo operations and free-up dollars for high value innovation that make the business better drives value.

  • Determine the strategic importance of IT projects to the business to:
    • extend capability,
    • expand by transforming and improve process or business model
    • extract constraints and release resources
  • Evaluate the evolving IT capabilities of competitors that could threaten your industry position
  • Allocate dollars across the portfolio of IT investments to ensure an efficient risk return?
  • Make  trade-offs in managing the IT portfolio
  • Effectively execute on major IT programs
  • Ensure protection from operational and security risks
  • Reduce investments in sustaining and increase investments in creating new capability


When you take a high cost on-premise service and make a better one available in the cloud for much less you drive value by extending capability and extracting cost and resources that can be applied more productively.

Excellence in Execution

Excellence in execution drives value. When 80% of all IT projects are failing to produce promised results, what’s wrong?  Most of the time, its flaws in execution. There are paradigm changing technologies available today that will drive cut the cost of development by 50% or more and deliver solutions to market 3 or 4 times faster than just a few years ago. To leverage them demands a high degree of excellence in execution otherwise few of these advantages will make it past a pilot and remainder result in over-budget, over-schedule failed projects. You can drive business value through excellence in execution leveraging the technologies that provide the most business advantage and seeing the process all the way through to full adoption and optimized use.

Rock-Solid Operations

A brilliantly designed system, impeccably implemented, with users training to extract optimal use from it is valueless if it’s not available when the business needs it. A system that leaks competitive or private information is also valueless. A system that cannot scale up and down to meet the demand cycles of your business is again valueless. You can drive business value by creating a certainty that a system critical to the business is always there when it is needed. It requires focus on process, on technology and on the people who are responsible for the continuous delivery of the service to the end consumer.

Check to see that you are truly driving business value in your engagements.

  • Is your solution or service “me-too” or are you creating unique  advantage for your client?
  • Is their traceable and verifiable alignment with your client’s business goals and objectives?
  • Are you extending business capability, optimizing or transforming the business model?
  • Are extracting cost and resources from the current environment that can provide resources to build their business in other ways?
  • Is your solution focused well beyond deployment to the actual adoption and use of the system where you can measure the benefits received?
  • Have you delivered a technical solution that will stay running, validated the operational processes with the full  trained staff?


Posted in Consulting Excellence, Uncategorized | Tagged , , | Leave a comment

What’s the cost of bad IT architecture?

Costa_Concordia_-_2006-07-04_Genova_-_19m CarnivalSplendor-Mazatlan

Look closely at these two cruise ships. Except for the name and the funnel, they are almost identical. In fact they are sister ships both owned by Carnival Cruise lines. Also in common, they both suffered major incidents, both incidents preventable with better engineering and both incidents causing damage to customers, the company and the cruise industry as a whole.

The Costa Concordia sank off the coast of Italy on January 13th, 2012 with tragic loss of life by colliding with a rock. A rock that was well known, existed on nautical charts for generations and on a route that this particular ship had traversed over 100 times. But somehow they still hit the rock. More significantly than that, the ship could not survive it. Why?


The Carnival Splendor made international news last year as the SPAM cruise. After a break in a crank shaft, a fire broke out in one of the engine rooms and caused a failure. A single failure that also disabled all other engines* and power generators on the ship, leaving it immobile and powerless in the Pacific Ocean, where food (Spam as it requires no refrigeration) was air dropped to the 3,000+ passengers while it was towed back for days to San Diego. Why?


Stuff happens. It happens on cruise ships and it happens in Data Centers. More than 80% of all Data Center service outages are caused by human error and in the other 20%, stuff breaks or fails to perform as expected. Good architecture whether that is Naval Architecture or IT Architecture anticipates issues, prevents them where possible and perhaps most importantly has a recovery process that does not sink you or leave you powerless and adrift.

Despite have multiple engines, the Carnival Splendor had one or more single points of failure in the mechanical design that completely disabled the ship. The Carnival/Costa Concordia had at least two fatal design flaws in navigation (control) systems and in  hull (resiliency) design. Should the architects of these two ships have:

  • anticipated the loss of an engine ?– yes
  • anticipated the possibility of human error in navigation ? – yes
  • anticipated that the ship could hit a rock and require flood control chambers ?– yes

Now look at your data center.

  • does your IT architecture anticipate and compensate for the loss of critical components?
  • does your IT architecture include the possibility of human error in operations and work to prevent, reduce and worst case compensate for the error?
  • is your IT architecture resilient? Can it bounce back from  a hit quickly and keep your business running?

Another thing also in common with Cruise Ships is that your IT architecture design decisions are best made at the point of initial construction. It’s going to be be very expensive to add those water tight doors to Deck 6 later, just as it is to add resilience features to your architecture once it is already in production.

What’s the real cost of bad IT architecture? Part of it is the business interruption and disruption of having a system down and the associated “cruise refunds” for your customers but the real cost is in trust.

When people don’t trust the service to be there when they need it and become concerned that the service may impact their safety or the safety of their business, they stay away. Today, how many people will be booking their Mediterranean cruise? How long will it take for that industry to recover?

The true cost of bad IT architecture is immense. Don’t cut corners. Hire great architects to design something that won’t sink your business.


(* Carnival Splendor  diesel-electric propulsion system consisted of two engine rooms with three banks of  Wärtsilä 1.3 Mw diesel engines each. 1 engine failure took out the other 5 engines. Engineering Report )

Posted in Consulting Excellence, Technology | Tagged , , | Leave a comment

Why Microsoft could be the key to prevent and cure Cancer

Technology Post

If you have not seen Richard Resnick’s “welcome to the genomic revolution” on TED you really should. It will inspire you and help you catch up with a medical revolution that will change the world and may just save your life. Allow me a few moments to provide some background, so that the topic of this blog post will not seem like quackery.

The human body consists of four key components nucleic acids, proteins, carbohydrates, and lipids. Nucleic acids are the ones that are required for creating, growing or mutating cells. Yup, that’s the DNA, RNA stuff.

DNA at its core is not too complex. There are only 4 components Adenine(A) , Thymine(T), Guanine(G) and Cytosine(C). Only AT and GC bind nicely to each other in pairs and is the main reason that figuring out DNA is actually simple.

1pair 2pair

The funky DNA helix chain is a sequence of our friends ATGC glued together. Each combo is a “base pair” (BP). So a sequence could look like something like this in the two strands:



Now due to some amazing technology over the last few years and the completion of the Human Genome Mapping Project , the medical world is about to quickly turn healthcare upside down if they get a little help from Microsoft.

Where the complexity of DNA comes in is not in the biology, there are only 4 basic elements; it is in the math. The cure will occur in 5 phases:

  1. Data Capture of the DNA information on a broad enough scale to drive statistically relevant data points. The high volume input of sequenced DNA evidence is required.
  2. Analysis that looks for patterns, mutations and can answer the question “what is involved and influences the cancer”
  3. The DNA and associate Genes need to be mapped to the Protein level where we can understand the role and the biological function of the protein in the process and what drugs or bio-process can influence the protein
  4. We then need to do drug and bio-process response modeling
  5. Develop clinical remediation options

There has been amazing progress in the technology to create a DNA sequence, step 1. The 30 second “Coles-notes” version of how this happens is this. They take the cells with the DNA and melt them where the double-stranded DNA unwinds and separates into single-stranded strands. Then by chemical reaction break apart the strand into chunks and through electrolysis sieve the small strands (1 nucleotide in size) into a gel. These are then split into 4 groups where 4 different chemical reactions attach a “marker” (either radioactive or chemical) to the four base acids (ATGC). The sample is then recombined with each channel and viewed with X-Ray/Ultraviolet light etc. showing the associated ATGC position in the sequence. This process that used to take weeks of effort for a small number of base pairs, can now be done at the rate of Billions of BP per day in a single machine by the latest technology.


So the bottom line is that today, you can have your entire genome mapped in a day, all 3 Billion bits of information and store it likely in your phone.

So now that we can produce vast quantities of sequencing data what does it mean? We now need to process this data. (To answer the process question we need to know a bit about genes.) A gene is “locatable region of genomic sequence”. Ie. When you look at a DNA strand, in some, many or no places on that strand there is a sequence of base pairs that defines a certain attribute. (Perhaps your predisposition to a certain type of cancer).

The total complement of genes in a cell may be stored on one or more chromosomes. The good news for humans is that we only have 23 Chromosomes.

However inside those 23 Chromosomes are a few base pairs, about 3 Billion of them.



Total base pairs



































































X (sex chromosome)



Y (sex chromosome)






In those 3 Billion pairs, we have some challenges.

  • The direction you “read” the sequence matters.
  • Groups of pairs have a sequence reading frame which means that depending on where you start reading the frame in the strand you get a different amino acid. For example, (borrowed from wikipedia) the string GGGAAACCC,
    • Every sequence can be read in three reading frames, each of which will produce a different amino acid sequence.
  • We need to able to recognize patterns in the strands and across strands. It would be great if the “Cancer” gene would just show up in the DNA labeled and in one place. However, whatever the causes or predispositions to cancer are, they are ALL OVER THE PLACE. (As an example, rearrangement of DNA between chromosomes 9 and 22 is associated with several types of leukemia) It could be across multiple strands, multiple chromosomes and perhaps even outside the chromosome as well.

Hence, why I would state today that the magnitude of this “math” and data processing problem is best solved by Microsoft.

Let’s simply assume that for the next 2 years that every person in British Columbia, Canada (population 4.5 million) who is diagnosed with cancer each year (22,000) provides a contribution of their DNA to a study but let’s just try to find the genetic patterns in the top four cancers. So out of the 44,000 DNA samples we have 27,500 covering:

  • 6000 Breast Cancer cases
  • 6000 Lung Cancer cases
  • 9000 Colon Cancer cases
  • 6500 Prostate Cancer cases
  • 27,500 New Cases of the most common cancers

So let’s do a complete sequence for every case (and perhaps family members as well) resulting in approximately 83 trillion base pairs or perhaps 10 times that if we pull familial and asymptomatic cell DNA as well. Now go look for patterns in those 830 trillion. There are only 20 amino acids that can be made from ATGC so the actual combinations to look for patterns drop to about 40 trillion if you guess right on the start sequence and 120 Trillion if you don’t. We can start at the macro layer and see if there are patterns in the 32,000 genes to simply the problem set but more than likely it will be combinations of patterns that will be the cause. Is it too complex? No. It is complex but it is finite.

Curing and preventing cancer is not far-fetched at all. Here is why:

  • Proteins are involved in everything cells do, including cancer processes.
  • Cancer absolutely has a genetic component
  • Drugs work by influencing the actions of proteins
  • Drugs are metabolized by proteins
  • The genetic components of Cancer CAN be mediated by drugs.

So what do we do then?

What is common in every step of the process is data. Masses of data.  Some is raw, some has metadata and some is in the form of research papers published from thousands of researchers over the years. It is both structured and unstructured and technology is required to turn data into meaningful clinical information quickly.

The scale of the processing required to address this type of problem set is massive. While I have forgotten most of my university combinatorics classes, I do know that the combinations and permutations of a 830 trillion piece dataset is not the job for calculator.exe on a PC. It will require massively parallel computing technology, tens of thousands of CPU’s, massive databases, data mining, expert systems and perhaps pattern matching technology yet to be invented for this purpose. Microsoft is the one company globally with massive cloud datacenters, High Performance Computing (massively parallel)/ Parallel Datawarehouse technology and research resources to not only make the next step in the cure possible, but to collaboratively connect the world’s cancer researchers to do it.

Microsoft research has already created and placed into public domain tools like FaST-LMM. (Factored Spectrally Transformed Linear Mixed Models is a program for performing genome-wide association studies on massive data sets.) Microsoft has the research capability to develop the necessary tools to not only process and analyze research data but also to pull together the existing knowledge of prior research results and help researchers to apply it in context.

For raw processing capability Microsoft has demonstrated > 1 Petaflop (A petaflop is a measure of processing speed and can be expressed as a thousand trillion floating point operations per second) running a single problem set cross more than 1200 nodes in the Windows HPC server. This is among the best in the world and will continue to go up.  It is the ability to scale to any data crunching requirement that will allow researchers to quickly investigate, model and test ideas.

The raw data generated from a single end to end DNA sequencing is about 7 Terabytes. Now multiply that by our sample group of 440,000 more runs. That’s 3 million Terabytes of data. To provide fast responses to research or clinical queries or expediting data mining looking for outliers, you need massively parallel database technology. Again , Microsoft has just brought this technology to market with SQL Server Parallel Datawarehouse.

Lastly, you need to run all this software somewhere. Believe it or not building out a secure datacenter that could house this amount and level of technology is not just a function of enough investment. A datacenter to host this would be in the 200 Megawatt power range. There are not too many places on the globe that have a spare 200 Mw available on the grid at any cost. However, Microsoft knows where they all are and has already been busy thinking about the next monster datacenter builds. I know for certain of a high security location in Canada that would be ideal and ready for a secure Datacenter build quickly.

Sure lots of other companies have a portion of the solution but Microsoft can actually bring all of the required components today and it engages 90,000 Microsoft people in the fight also. Nothing inspires or drives performance more than a big challenge to do something that will change the world. (again)

With a little encouragement; Microsoft could be the next critical piece of the puzzle to help the thousands of researchers prevent and cure Cancer.

Posted in Technology | Tagged , , , | 1 Comment

Reality Bites in Consulting


An optimist, a pessimist and a cynic went into a restaurant. On the table waiting were three partially filled glasses of water. The optimist said “look my glass is half full! “and took a drink. The pessimist said “Look my glass is  half empty!” and took a drink. The cynic said “my glass is probably filled with toxic city tap water” and refused to drink at first but then relented and tentatively took a drink.  The waiter came by and said “Here let me clear those dirty glasses out of the way for you, wipe the table and get you some menus”.

A client does not benefit from a consultant being an optimist, pessimist nor a cynic. An excellent consultant is just like the waiter …  a realist.

One of the most common errors that consultants make is to gauge the success of the engagement by the momentary happiness or grumpiness  of the their client. The client may be too optimistically viewing the engagement, oblivious to its challenges or a client may have unwarranted pessimism. Both of these positions can potentially provide an incorrect reading which in itself is harmless until the consultant uses it as their own barometer of engagement success.

All consulting engagements have a  lifecycle and in the lifecycle there are inflection points where the complexity, risk and quality of execution varies.  Experience yields a well-worn pattern for most types of engagements.


I have seen too many consultants take their clients to the proverbial peak of a mountain during a blueprint stage and thrill them with a future view from the executive charter helicopter. The view from the finished building on that mountain peak will be stunning! The consultants rarely mention the challenges of dragging the building materials up the side of the mountain. While it is not critical that the client understand the project lifecycle and tasks in detail, it is certainly important that the consultant does and also for the consultant to make sure they are  able to communicate to their client that the initial helicopter ride was just that… sightseeing.

Project reality bites. Depending on how realistic you have kept your plans and communicated those plans to your client, the bites will either be small nips that don’t cause major changes in the planned route and speed or they will be  full-on wounds that require intensive care and leave a lasting scar on your client relationship.

It’s great to have a happy client early in a project but true success depends the consultant’s own assessment of the current state and future project challenges to make sure a realistic plan is in place and the client is pleased with the final outcome not just the beginning.


Posted in Consulting Excellence | Tagged , , , | Leave a comment

Does Your IT Vendor have their head in the clouds?


So why do clouds exist? The answer is that we always really wanted them but they are just now technically possible. Nobody ever thought that having as many servers as possible, geographically disbursed under people’s desks, unmanaged or mismanaged was a good idea. It wasn’t. The issue was network. We wanted them close to us, because the network and/or the systems were unreliable or too expensive. We wanted that nice fast tactile and visual performance of the LAN attached server and we wanted to able to reach over and power it off and back on again when it misbehaved. When the networks and management tools improved, we pulled the servers back in from the field and dropped them in a central data center where they could at least be managed by professionals, reliably backed up and in theory provide better service than sitting locally.

Now each company has its own servers, uses a variety of technologies for application and management and employs a staff of experts to keep the environments running. The improvements in tools now let them look after a higher quantity of servers, but the servers are still a mish-mash of technologies.

Then standards arrive. By establishing standards for operating environments, application development, middleware, database, identity, provisioning, network, management and security suddenly the ability for tools and a few skilled resources to manage servers jump from hundreds/thousands to hundreds of thousands of servers. This creates immense economies of scale savings in operations.

From there, now add specialized hardware where application servers and database servers are actually engineered to provide specific performance characteristics at the lowest possible commodity hardware and operational costs. You then add and manage near infinite storage with dark fiber replication across sites instead of operational backups; again eliminating cost.  When components break you now literally unplug the server, (the load having already been shifted to an identical server) and throw it away as repair logistics cost now exceed the cost the replacement.

So whether the cloud is internal or external (private or public), the approach can dramatically drive down the operational costs of an IT solution. The public clouds add the potential benefit of reducing capital expenditure as they are mostly offered on a pay-as-you-consume model.

So why does your IT vendor have their head in the clouds?

The short answer is that they know that ultimately server-based application processing will become a complete commodity. Will you care in a Cloud model whether your database is SQL Server, DB2 or Oracle? The answer is no you won’t. You will have an expected service level, expected performance level and require X amount of storage, but you won’t care the label on the database. It will have become a complete commodity. All of the commercial databases do the basics, very, very well.

Now does that worry Larry Ellison? Yup, 67 year old Oracle founder Ellison will need an extra bottle of Grecian Formula to scare away the grey hairs when thinking about this problem. Why, because as customers continue to build applications on Microsoft Azure cloud infrastructure, they aren’t even asked about which database they would like. SQL Azure (AKA Microsoft SQL Server) is the only choice provided and it simply comes with the cloud service. (and it’s a great database)

Microsoft has made a few strategic boo-boo’s over the years; being late in the web browser game and later still into phone mobility but they sure didn’t make the same mistake with the cloud. They recognized early that to be a player in Cloud is only for the biggest of players. At the core is a winner take all approach in the platform-as-a-service (PaaS) market. (the other cloud models by the way no vendor is really interested in … see my post on everything-you-need-to-know-about-the-cloud-in-one-short-blog-post for an explanation)

To play in PaaS, not only do you have to build and operate monster datacenters around the world, you need to create a complete application development, runtime and management stack to run on those specialized datacenter computers at a cost of many, many billions. It’s all about economies of scale.

Why does your IT vendor have their head in the clouds, it’s because the industry has inadvertently created its own killer and set it loose.  It has commoditized its high cost specialty products and the only vendor left standing will be the one that achieved sufficient scale and volume to stay in business.

Don’t believe it?

How many PC manufacturers are there left?

Posted in Technology | Tagged , | Leave a comment