IOB Followup

Last month I read a piece in CLO magazine by Blackboard's Timothy Hill on what he called Impact On Business, or IOB. I was able to make contact with Mr. Hill to ask him further about his article. His office responded very quickly and openly to my request, sending me a white paper he had put together on the concept.

The white paper doesn't discuss anything new, although it is a bit more specific about what IOB means in a very general sense. A short discussion of the growing importance of informal learning and the technologies that facilitate it in the workplace, the importance of working with customers to modify learning objectives and strategies to meet them; these and other well-known tenets of a successful training group are acknowledged. ROI is mentioned a few times early on, but is never revisited in a meaningful way. IOB as a concept is again not addressed in any meaningful detail.

Now that I have completed my ROI class and know quite a bit more about ROI as a practice, I find it hard to assert, as Mr. Hill does, that ROI is ineffective in measuring learning in the workplace. After all, Level 2 of the Kirkpatrick model is designed specifically to address this component, and coupling Level 2 with Jack Phillips' Level 5, or ROI, gives the learning piece the sort of hard data connection Mr. Hill says training organizations need. In fact, the way the concept of ROI is treated in the paper appears to imply that ROI is a somewhat simple evaluation step to complete at the end of a project. After having just finished a class about ROI, I can say without question that ROI not only covers everything Mr. Hill talks about, but is also not quite so short and easy a step. It's a long and detailed process.

After reading the paper, I have to say that the IOB concept was little more than a device to sell Blackboard's Learn+ product, as evidenced by page 8 of the PDF. While white papers are certainly used for sales purposes, I was again left unsatisfied in reading about an alternative perspective to ROI. I also take a bit of umbrage with the assertion that ROI cannot handle measurement of learning. Quite the contrary, the complete ROI methodology covers evaluation of not only all four of Kirkpatrick's levels, it describes a total process. From planning evaluations, to designing measures, to collecting and analyzing data, to determining a final result with bottom-line goals factored in, and communicating results to stakeholders and management, every important evaluation activity is included.

Not to evangelize here, but Mr. Hill's IOB concept misses the mark in a bad way. He seems to be painting ROI as flawed and a bit passé, while recasting what his audience wants as something else that ends up totally undefined. Creating new acronyms and vocabulary doesn't help if the audience is already oriented to another standard. Playing to that entrenched methodology would also speak directly to the audience, as opposed to confusing them. His point could have been made more correctly by describing how Blackboard Learn+ could aid in conducting ROI (specifically as a data repository). And on that note, anything that provides more hard data will be more useful in an ROI evaluation. Perhaps Learn+ is such a tool, but the false acronym under which its sale is marketed only confused me as a potential client.