Category Archives: Consulting

The Big Data Gold Rush

As storage costs plummet, NoSQL databases mature, and processing power continues to follow Moore’s Law into the stratosphere, companies everywhere  are talking about the potential of Big Data to revolutionize business, unlocking hidden value and allowing companies to gain a competitive edge.  If you ask the vendors and consultants involved in this space, every company is sitting on a gold mine of untapped data, just waiting for a clever statistician with the right tools to dig out its value.  But is the whole big data push just hype?  An analyst at Seeking Alpha seems to think so.  First, he breaks down the six sub-hypotheses underpinning the big data impetus:

A) This data collection is a recent phenomenon B) Value is not already being extracted from whatever data is being collected C) Companies will need outside help to extract insights D) Outsiders can help companies extract insights without having deep industry knowledge E) The insights gathered from ever larger data sets have more value and are more accurate than insights gathered from smaller data sets F) Unstructured and cross functional data have huge value waiting to be extracted.

His final conclusion after examining each one?

My personal opinion is that the potential market size and value proposition are both over-hyped. I’m reminded of the hype over biotechnology in late eighties and early nineties. For example, biotechnology was supposed to have brought us juicy red, delicious and stackably cubical tomatoes that would yield square slices that fit a slice of bread. I’m still waiting for my square tomato slices.

In fact, when you look at the “success” stories of companies such as IBM, the ballyhooed results are almost comical (“Reduced Report Generation Time by 63%!”). Another common theme, found on all the data analytics companies websites, is that their focus areas seem to be internet, banking, healthcare, industrial, life sciences, CPG, insurance – the very areas which have traditionally crunched large amounts of data before it became “sexy”.  So where are the big inroads into new markets? One example of such an inroad is law. Law firms’ revenues have beendestroyed by revolutionary data analytics technology, which has replaced law associates with software, leading savvy clients to pay very much less for the same work.

The law firm example is especially apt for this blog, and hopefully I’ll have time to dive into the world of big data in law another time, but for now – I’m even more interested in his point about magic sandwich tomatoes from the early 90s.  The more I hear consultants touting big data as the next big thing, the more it sounds like the exact same language they’ve used to describe any sort of deep analytics over the past ten years.  Big data isn’t new, it’s a repackaging of the same old.

Consider this McKinsey report, widely cited in the industry, which may as well have been written in 1999 and updated to change gigabytes to terabytes or petabytes.  Big data to revolutionize retail?  Wal-mart, Target, and other big-box stores have been doing it for decades – that’s the whole reason grocery stores have loyalty cards.  The big data techniques for marketing have been used for decades in order to improve the effectiveness of direct mail.  In many corporate functions, like supply chain and HR, the promises of Big Data are the same as the promises that enterprise resource planning providers like Oracle and SAP have been touting for years.

So if Big Data is all hype, is there any gold there?  Underneath the surface, there’s still some opportunity.  First, the returns to rudimentary analytical skills continue to grow.  We’re reaching a point where innumeracy in the business world is becoming an even greater handicap, and the ability to analyze and interpret data will take a professional a long way.  Second, analytics is democratizing.  Many sources of big data are being made public, or available at low cost, on cloud computing servers, through services like Twitter, or by academics.  The more eyes looking at data, the greater the likelihood someone will find something revolutionary, which is why Kaggle competitions offer so much potential.    Third, the de-anonymization of the internet has the potential to allow marketers to link together a wide variety of consumer data (of course, with worrying privacy implications), which could open new opportunities.

So while I don’t think Big Data is the Next Big Thing, it’s still the continuation of an exciting trend that’s helping businesses become more effective and more efficient.   And when consultants find a new label for “crunching your numbers to help you find insights,” touting it as The Next Revolution For Business, I’ll probably use this same post again, and update my petabytes to exabytes.

What’s the difference between a researcher and a consultant?

My first mini web comic slide:

What's the difference between researchers and consultants?

Why do smart young people go into finance, law, and consulting?

The economists over at George Mason had a discussion this week about professional services firms (it looks like consulting especially, but not exclusively) that led them to try and answer two questions of interest to this blog: why do smart motivated young people join professional services firms, and why do firms hire consultants.  I’m going to try to tackle both questions with my thoughts, so consider this part one of a two part series.

First off, Tyler Cowen offers his theory on why so many smart young people are going into professions like finance, law, and consulting:

The age structure of achievement is being ratcheted upward, due to specialization and the growth of knowledge.  Mathematicians used to prove theorems at age 20, now it happens at age 30, because there is so much to learn along the way.  If you are a smart 22-year-old, just out of Harvard, you probably cannot walk into a widget factory and quickly design a better machine.  (Note that in “immature” economic sectors, such as social networks circa 2006, young people can and do make immediate significant contributions and indeed they dominated the sector.)  Yet you and your parents expect you to earn a high income — now — and to affiliate with other smart, highly educated people, maybe even marry one of them.  It won’t work to move to Dayton and spend four years studying widget machines.

You will seek out jobs which reward a high “G factor,” or high general intelligence.  That means finance, law, and consulting.  You are productive fairly quickly, you make good contacts with other smart people, and you can demonstrate that you are smart, for future employment prospects.

That’s one theory, but I think it only plays into one motivator for a much more complicated decision.  I think Tyler is failing to appreciate just how intimidating graduating from college and choosing a career is for anyone without a technical and focused degree like engineering or accounting.   Up until you graduate college, each step in life is scripted – you go from middle school to high school, and from high school get good grades to apply to college, where you select the best college that you can get into.  All colleges are fairly similar in the sense of what you’ll get out of them is a degree, and all ask for similar inputs in the sense that they really want to see grades, test scores, and extra-curricular activities.  They don’t require you to declare a specialization or have any focused interest, so you can head off to explore the world without the terrifying possibility that you may close off some door to the future.  Law school lets you continue this process for another three years while staying under the comforting protection of the academy.

But applying to jobs is hard.  For starters, unlike colleges where you have a finite number of options conveniently ranked by status on a handful of similar lists, there are tens of thousands of employers in any given community, let alone the country, and the lists that rank them aren’t nearly as thorough or complete (if you ever want to feel good about college rankings, check out the methodology for a best place to work list).  Compare that to law, finance, accounting and consulting, all of which have a smaller number of firms that hire people straight out of school, and which are conveniently ranked by prestige by their own trade publications and sites like   In other words, the professions have their own equivalent of the US News rankings to help college graduates decide.

Not only are there tons of firms offering potential employment in the rest of the economy, but finding them is really hard.  They usually publish jobs on their own company’s websites, maybe the occasional job board (although for professional positions, most recruiters I know don’t like them), and through field specific headhunters.  In other words, even for entry level jobs, they want the new graduate to find them, not the other way around.   Professional firms show up on campus, throw giant parties, give presentations on how their application process, and even host the interviews right in the business school.  It doesn’t get much easier than that.

Finally, when it comes time to apply, all the professions are looking for pretty much the same thing, and it’s happily similar to the college application process that you’re not only familiar with, but have demonstrated that you’re pretty good at.  Fill out your GPA and test scores, write an application essay, upload your resume full of extracurriculars, and get ready for a few years of Excel, PowerPoint, and doc review.  Compare that to non-professional firms that want you to demonstrate your ability to do the job, despite your negligible experience, or tailor your cover letter to their culture when you barely understand what work is like.

In some ways, this supports Tyler’s point – these firms are excellent ways to quickly attain status with low risk.  But put another way, they’re also the easy choice for a generation of college graduates that’s never really had to choose, never had to explore what’s out there, and isn’t ready to settle down and make a career.  Some of this may sound condescending or bitter, but in actuality this process is hugely beneficial to the firms that hire these new graduates, and more importantly to those firms’ clients, as I’ll try to cover in Part 2 when I look at why companies hire consultants.

Supply Chain/Operations Case Study, Courtesy of Jim Henson

How could a business case study get better than this?  Building on the previous two posts, as well as my other passions, this 1963 Bell Data case study features:

  1. A business consulting case (operations/supply chain/communications)
  2. A Jim Henson Muppet
  3. A robot
  4. Rocket ships

A great example of each.

Enjoy, via @RobotCity_Chi

How Elite Firms Hire

Bryan Caplan is linking to a new study that covers an oft-discussed, but poorly researched, area of professional services firm management: the actual hiring process at elite firms in law, finance, and consulting.  (The actual paper by Lauren Rivera, “Ivies, Extracurriculars, and Exclusion“, Research in Social Stratification and Mobility 2011, is behind a paywall).

Bryan outlines the five key highlights:

  1. Most applications practically go straight in the trash
  2. Evaluators have a lot of slack
  3. Super-elite credentials matter much more than your academic record
  4. Super-elite schools matter because they’re strong schools, not because they’re better at building human capital
  5. Extra-curriculars matter

Ask anyone at a major firm in finance, law, and consulting, and you’ll here about the dedication that every professional has to the hiring process, that because talent is the cornerstone of any successful firm, everyone up to the partner level is heavily involved in recruiting and selection.  You’ll hear paeans to diversity in the firm, not just in terms of race or ethnicity, but also backgrounds and interests.

The truth, as Rivera reveals, is more mixed.  Professionals at all levels are indeed involved in hiring, but they’re also busy with client work, and resume reviews are quick skims. And despite practicing professions dedicated to rationality, they’re still humans who suffer from the same hiring biases as any other manager.  While they talk about diversity at recruitment events, they look for signals of people who are just like them, especially when it comes to attending an Ivy League university, which anyone who has tried to break into a top-tier firm from a school off the standard recruiting schedule can tell you.

As Megan McCardle points out, this isn’t just bad for society, it’s terrible for the firms:

Forget about the effects on society, though; this is terrible for organizations.  You see this in Washington all the time–a friend who went to a lesser-known state school said he could always tell the people he wasn’t going to like when he met them at cocktail parties, because the minute he told them where he’d gone to school, they became extremely interested in going to get another drink or find the cheese dip.  This is one of the smartest, most consistently interesting and original, most talented writers I know.  Having actually attended one of those elite schools that apparently make you fascinating, I can attest firsthand that statistically, the elitists were vanishingly unlikely to be as interesting as the person they abandoned because he’d gone to a state college.


The Ivy League is full of smart, interesting people.  But it is not full of all of the smart, interesting people in the country, or even a majority of them.  And given the resumes required to get there, it produces a group of people who are narrow in certain predictible ways.  (I include myself in this: just because I can see it operating doesn’t mean I can escape it.)
The problem is that actually seeking out a wide variety of graduates would be much more expensive and time consuming.  Why spend the effort searching for “best” when you can easily access “very, very good”?


It’s not actually a great personal tragedy to be turned down by McKinsey (she said, from personal experience); there are still a lot of interesting and remunerative jobs out there.
But it may well be a corporate tragedy for McKinsey and its clients.

The homogenization of top-tier professional firms might not be as bad as Megan makes it sound.  At their core, these firms are looking for people who are willing to work extremely long hours, with intense focus, on work that might be analytically demanding, but isn’t usually that interesting.  Not coincidentally, that’s what elite universities usually screen for too, so university admissions serve as a useful shorthand.  It’s also why many of these firms are finding new sources of talent in burnt out PhD students – what better way to prove that you’re willing to work really hard at something you don’t enjoy than spending five years on a degree you don’t even want?

Rivera’s finding about extra-curriculars, while it surprises Bryan, lines up fairly well with how I’ve heard professionals describe “diversity” in their firm.  If you listen closely, diversity falls into one of two categories.  First, there’s diversity-of-academic-background, which means that there are people running around who went to the same elite institutions, but got a degree in something other than business or economics.  The second is diversity-of-outside-interest, and usually involves the anecdote about the partner who’s in an improv group, or the managing director who likes to build custom motorcycles.

These people are interesting in the sense that they’ve shown a fanatical devotion to at least one thing outside their core practice, but I’m not sure if it’s what I’d rely on to bring intellectual diversity to my firm.  And it’s intellectual diversity that leads to better outcomes for clients, as  Scott Page discusses in his book The Difference (I wish I could quote from it, but my copy is in storage in Detroit).

In contrast, I’ve seen a number of companies, both professional and corporate, that practiced what I call”talent arbitrage,” recruiting the best and brightest from top-tier state universities that fly under the radar of the usual hiring elite .  GE used to be well known for this practice, but I’ve also seen it more recently in rapidly growing software companies, profitable legacy manufacturers, and highly successful specialist consultancies.  It takes a much more sophisticated human resources function and dedication to talent management than most companies employ, but it’s also a way to hire great people without having to offer the astronomical salaries now needed to recruit from the Ivy League.


Monitor Group Under Fire for Qaddafi Project

Monitor Group, the top-tier strategy consulting firm, is in hot water for their relations to Libyan President Muammar Qaddafi, who is currently engaged in a brutal war of repression against his own people.  Mother Jones has the scoop on the project:

But the firm also succeeded on other fronts. The two chief goals of the project, according to an internal document describing Monitor’s Libya operations, were to produce a makeover for Libya and to introduce Qaddafi “as a thinker and intellectual, independent of his more widely-known and very public persona as the Leader of the Revolution in Libya.”

I’m confident that Monitor isn’t the only major consulting firm that engaged with an autocracy to help them gather legitimacy on the international scene, they’re just the guys who got caught when their client decided to respond to peaceful protests with a military crackdown.  

Given the criticism Monitor is now receiving for their work, will it make other consulting firms more hesitant to take on dictators and their governments as clients?  Or will the financial appeal, combined with a genuine belief that they’re helping these countries move towards more democratic government and efficient services for citizens, keep consulting firms engaged?  At the time, I don’t think anyone at Monitor thought there was a problem with what they were doing, but I have to think there’s some serious soul searching going on at the firm right now.