Online Versus Paper Assessments: What's the Best Approach?
by Roberta Hill
I am always surprised when others, including coaches, fail to share my excitement about online assessments. However, there are good reasons to be skeptical. My mother's childhood expression, "What you lose on the swings, you gain on the roundabouts," is, I think, an apt summary of the tradeoffs between paper and online assessments. For purposes of this article, I will consider online and computer-based instruments together, since the same principles apply to both.
Despite the hype and excitement about online assessments, paper versions continue to remain more popular. Here are a few reasons why:
Getting everyone to complete an assessment online can be an administrative challenge. Working in groups requires a strong and well-structured communications strategy, so this issue must be sorted out on an individual basis before offering the tool to your client.
Despite my obsession with technology, I have discovered that my enthusiasm is far from being shared by the general public. Even in the business world, I find that about one third of the people I deal with neither like nor trust assessments—particularly online ones. They doubt both the confidentiality and the credibility of the process.
Although technology seems great, emails and reminders are often forgotten, or they can end up in 'Junk' folders. Company firewalls can block emails and/or prevent client access to the internet site that holds the results of their assessments. And, believe it or not, not everyone has a personal email account or internet access at home.
In my view, there are only two 'real' but significant drawbacks to paper versions: Inventory and time. When purchasing paper assessments, a good unit price usually depends on buying a minimum of ten instruments. When I cleared out my inventory of various hard copy assessments last year, I discovered that I had close to a thousand dollars worth of retail material for the Myers Briggs Type Indicator that I will never use. Some of it was dated—I had various versions or editions of material that I could only administer in one-on-one situations. In addition, I had approximately another thousand dollars invested in other types of assessments.
While it takes about the same amount of time for a client to take an online or paper assessment, the administrative process is more cumbersome with a paper version. It takes time to deliver the assessment to the client, collect it back, score it (which can be quite labor-intensive and time-consuming if the instrument is complex), and return the output for client review. Conversely, when using an online assessment, distribution to the client, data collection, scoring and response to the client are automatic and seamless.
That said, I have encountered the following drawbacks to online tools:
25% of clients fail to complete them by the due date;
The instruments are often limited to the English language;
As with paper versions, virtual 'inventory' or credits can be a problem. Purchasing a distributorship or even opening an account often requires the purchase of materials that you don't want and never use;
While the unit cost appears to be much lower than unit cost for paper, there can be hidden charges, such as annual access fees, which result in equivalent cash outlay over time;
Not all of the online systems are terribly user-friendly—either to the client or to the administrator. Lengthy logon systems, difficulty in accessing the reports, and other technical issues can be problematic, and not all systems have reliable Help Desks; and
Finally, there is a tendency for coaches/administrators to rely too heavily on the report, letting it take the place of proper introduction of the model behind the tool, adequate debriefing, and resultant dialogue between client and coach.
It is far too easy to send out the information for an online assessment and then let the process take care of itself. If you are administering a paper instrument, you need to 'know your stuff.' You must be there to answer questions, explain the tool, review the material and discuss the results with your client. There is no fancy 30-page color report to fall back on. Instead, you personally discuss what the information means to the individual. An automated assessment often takes on a life of its own!
The level of training and certification required to administer online assessments seems to be somewhat inconsistent. While this may or may not be true, it often appears that anyone can purchase an instrument or a 'knock off' online. This tends to be less common with paper versions.
On the other hand, instruments that can be taken online can be perceived as more 'valid' than paper versions. This is simply not the case. Psychometric rigor is even more of an issue with automated systems. Just do a Google search on 'personality tests' and you will find thousands of free quizzes available. There are many companies that provide survey and checklist creation services. Unfortunately, anyone—and I do mean anyone—can add a personally created 'assessment' to their website. While these informal inventories can be great tools when used properly, they are not legitimate assessments. Consequently, clients have good reason to be somewhat wary.
When it comes to using multi-rater assessments, online instruments win hands down. Computer programs can instantaneously perform the calculations that take hours to do by hand. Graphs of all sorts can easily be created to depict the results. However, I often think that the growth of the 360-degree market, and its prominence in the business world, is due more to technology than to good Human Resource Management strategies.
In summary, I continue to use paper in many team settings. However, with most of my business now being conducted virtually, I find that my needs are best met by online assessments. Besides, I am a techie assessment junkie! The ultimate question really is, "Will this particular online assessment meet the needs of my clients?" The answer to this question will always determine my recommendation.
Roberta Hill, MBA, is a Professional Certified Coach (PCC), as well as a Professional Mentor Coach (PMC) and Certified Teleclass Leader with Corporate Coach U International. Roberta owns www.AssessmentsNow.com, an online assessment provider with a network of more than 40 qualified coaches worldwide. Read more about Roberta in the WABC Coach Directory. Roberta may be reached by email at email@example.com.