FAQ: Understanding the ARC Scores

Read the answers to some of the most frequently asked questions about this year's Annual Report Card scores. If you have any additional questions, please contact me, VARBusiness' editorial research director, at bmarkowi@cmp.com.

What is the Annual Report Card (ARC)?
The ARC is a gauge of solution providers' level of satisfaction with their vendors from year to year. VARBusiness' ARC has become the industry standard for recognizing outstanding partner programs in product and service categories, from storage to network servers to security management software to business-class IP services--and more. "This year's ARC winners represent the top firms to look to for technology innovation and solutions in the digital economy," says Robert DeMarzo, vice president/editorial director of VARBusiness.

Who participates and how are the 2001 ARC winners determined?
Now in its 16th year, the ARC is based on a survey of 5,000-plus solution providers, who evaluate more than 70 vendor programs in 15 product and service categories. Participating vendors are selected based on market-share data from industry analysts. VARBusiness notified each participating manufacturer, asking them to submit a list of their solution-provider partners to be surveyed. Bernett Research Services, a Boston-based marketing research company, was retained to administer the ARC survey. In the course of the survey, solution providers rated the importance of 11 criteria in three key areas: products/pricing, support and partnership, and then rated their satisfaction with their vendors in each criterion. Overall scores were weighted to reflect the relative importance of each criterion. Scores were rated from 1 to 100, with 100 being the most desirable score. Loyalty scores were utilized to break first-place overall ties when necessary. This year, loyalty was factored into the weighted totals of the display technology and enterprise operating platform scores.

What score will generally take home an ARC award?
Commonly, a fair score in excess of the norm may very well earn a vendor the top spot in VARBusiness' ARC. In 2001, the average winning ARC score was 77, four points above the mean score of 73. This year's average score is consistent with the overall averages for 2000 and 1999 (73 and 75, respectively), as is the winning score for 2000 and 1999 (78 and 77, respectively).

In 2001, hardware categories were somewhat more likely to attain higher overall ARC scores. This year's hardware winners produced an average rating of 78, while software vendors won with a mean score of 77. ViewSonic earned the highest hardware score (82) for display technology, followed by Cisco (81) for voice and data networking, and IBM (81) for its iSeries midrange server. Among those receiving the highest overall software scores in 2001 were Novell (84) for enterprise operating platforms, Microsoft (77) for database software, and Veritas (76) for storage management software.

What is the most important measurement in the ARC?
Solution providers in the survey unanimously rated product quality/functionality as the most important criterion in this year's ARC, indexing 14 points above average. Overall ARC scores are weighted to reflect the relative importance of each criterion. Traditionally perceived as reliability, VARBusiness' 2001 ARC definition extends reliability to design, serviceability, performance, scalability, ease of use, flexibility, manageability and ergonomics.

With a score of 82, quality/functionality captured the highest average measurement score in this year's ARC. IBM's quality/functionality score in midrange servers illustrates the significance of the criterion. IBM's iSeries attained this year's peak ARC score in product quality/functionality with a rating of 91. Furthermore, in two out of three occurrences, a win in quality/functionality transforms to an overall category win.

What are some of the perennial trends in the ARC?
Perhaps the most notable trend in recent years has been the consistency of high scores in the networking arena. In three out of the past four years (1998-2000), networking scores have accounted for the highest overall ARC scores, ranging from 82 to 83 points. The overall victor has traditionally earned high marks in presales and post-sales support, as well as channel strategy and ease of doing business. Once more, Cisco clinched its second consecutive win this year. Cisco attributes its victories to sharp support and partner programs.

IBM's iSeries has garnered the highest overall score in midrange servers for four of the past five years (1997, 1999-2001), and ViewSonic has won five successive victories in Display Technology.

With this year's leading overall score, Novell placed first in enterprise operating platforms with a score of 84, back up from second place a year ago after winning the category in 1999. Novell credits its success to product quality/functionality, product availability, channel strategy, e-business programs, ease of doing business as well as to a lofty loyalty score.

Of note, Hewlett-Packard, winner of four overall ARC honors in 1999 (desktops, workstations, entry-level servers and systems management software) fell from the winner's circle this year. HP attributes its absence to a common lack of marketing support as well as to weak channel strategies, e-business programs and solution provider program administrations.

Which criteria are most likely to correlate with ARC wins and losses?
This year's wins may be attributable to robust channel alliances. A sound vendor program more often than not translates to access to much-needed support and services, timely communication, conflict resolution, well-defined e-business programs and ease of doing business. In nearly nine out of 10 instances, the vendor that won the Partnership subcategory award also went on to take the No. 1 spot in the overall category. Examples of such wins include IBM for entry-level servers, Veritas for storage management software, Macromedia for application development and deployment tools, and Seagate for disk drives.

On the flip side, Red Hat, a contender in enterprise operating platforms, fared poorly in the past two ARC contests. Its last place scores may likely be attributed to dissatisfaction with the vendor's revenue/profit potential as well as to a scanty partnership program.

Markedly absent from this year's winner circle, Dell Computer fell from first to last place for notebook computers. Dell's slide may be attributed to low scores in presales and marketing support, as well as to a void in channel strategy.

Are there demographic similarities among this year's ARC victors?
Solution providers behind the top winners in this year's ARC are significantly more likely than the average VAR to be reselling, recommending or supporting their vendors for five years or more (66 percent compared to 51 percent, respectively).

As for primary business activity, solution providers supporting the best-rated victors are more apt to be total-solution providers than ARC respondents at large (45 percent vs. 32 percent, respectively).

Solution providers of the highest rated vendors also appear to have greater annual revenue than typical resellers. Their median annual revenue of $7.6M is more than double the norm ($3.2M). Bucking that trend, in network storage, Sun's median solution provider income at $4.1M is lower than the category norm of $6.8M.

Who won this year's ARC awards?
Sun Microsystems received the most praise among participating vendors, winning two out of five overall category awards, and 10 out of 15 subcategory awards. Macromedia and IBM each won two category awards. Seagate garnered its first ARC win in disk drives, and Veritas, a newcomer to the ARC competition, received accolades, both for the storage management software category and for Breakthrough Player of the Year. ViewSonic won its fifth overall category award in display technology, as well as a Lifetime Achievement Award. Other overall winners were Compaq and Cisco in the hardware categories, and Novell, Microsoft and Tivoli in the software sector. Although AT&T received the strongest loyalty score, WorldCom took home a victory in business-class IP services for its overall satisfaction score.