In a recent piece on his own blog, Jeff Matthews points out the folly of paying too much attention to various white papers by supposedly neutral IT analysts working for companies like Gartner Inc. (NYSE:IT). I responded based on my own experience, but thought it was worth repeating the discussion here, with a bit of extra material, since it's relevant to the focus of this blog:
I am in the position of having been on both sides of Gartner. As a product manager in the software biz, I was a customer of the consulting services they offer to vendors. As IT management on the "buy side," I am and have been a customer of the consulting, advisory and research services they offer to the business customers of the very same vendors. At present I'm involved in at least one project in which a Gartner consultant is engaged.
My experience with Gartner, Delphi (who they purchased), Forrester, IDC and the others suggests that none of these companies is impartial, and that any "white papers," speeches at conferences or other materials provided by their analysts must be taken with a huge grain of salt.
The thing that you have to keep in mind about all these companies -- and which many IT managers either forget or never really understood in the first place -- is that they sell services to both sides of every IT transaction and do so in a self-serving manner. The vendors who spend the most on "consulting fees" to Gartner and the others tend to be the ones whose products are most often mentioned and recommended in white papers, speeches, articles and other public information generated by those firms. As a result, you see Gartner analysts go against the interests of their top vendor-clients about as often as you see Wall Street's Finest put "sell" ratings on their companies' investment banking clients.
Had I seen this kind of thing only once or twice, I'd be content to write it off as an abberation, but I've seen it consistently, both as a "buy-side" and a "sell-side" guy in IT.
A good example of how bad things can get is from a startup software company I worked for shortly before the boom went to bust in 2001. At the time we were one of five companies in our space, the best capitalized of the bunch, and the only one which had development formal partnerships with two of the major enterprise software companies. We were a bit slower in getting to the market, but arrived with an unparallelled featureset to go along with our strong partnerships and quickly became the #2 in revenues (not that that was saying much).
At one of the first conferences to address our space, the software analyst who claimed to be the leading expert (I'll leave out which of the firms he worked for) didn't even mention us when discussing the competitive landscape. When, during the Q&A, he was asked whether any of the companies in our niche had major software company deals (we were the only one that did, and the fact had been widely advertised), his response was that "he didn't know of any."
So we did the only thing we could. We hired his firm for a "vendor consulting engagement" with us, to "learn how to better position our company and products." We spent a day or so with that same analyst, showed him some demos (the same ones he could have seen on our website), introduced him to the CEO and a few other key people, and provided him with our own internal "white paper" (it was written by me and two other marketing guys) which outlined our own highly self-promotional view of the market and our company's place in it.
Within a few weeks, the analyst suddenly "discovered" us, published his own "white paper" that seemed eerily similar to the one we had given him just a few weeks before and began to mention our name to the press. At the next conference he spoke at, he highlighted us as the "leading company in the field" and went out of his way to highlight the advantages of our rather unique technology and approach.
Shortly thereafter the bubble popped, the company ran out of cash and sold itself to one of the larger software partners (only one of the other competitors survived in any form) and I moved back to the tech "buy side," where I have since made regular efforts wean my customers and employers away from overpriced Gartner, Forrester, IDC and similar "analysis" subscriptions. My feeling is that nobody should pay to read what is essentially other companies' advertising and that it should be read with extreme caution even if it's free.
I would be remiss if I didn't note that the practice seems to have abated somewhat since the boom years of the late nineties, which is when all ethics and morality seemed to disappear both on Wall Street and in IT. I'll also say that in recent projects I've found Gartner consultants to be fairly good and reasonably objective guys when retained for a specific project with specific parameters and scope, though like all consultants, they tend to recommend what they know best.
Still, a consultant who is brought in for a fairly well scoped effort and paid for by a client who reasonably expects him to have their own interests in mind is one thing. An "analyst" who publishes information for the general public or for a large body of subscribers is going to have a lot of leeway to take other concerns into account when he makes recommendations. As a reader/consumer of this stuff you've got to ask yourself: "Who is he really working for?" Oftentimes, it's not you.
-btc
