by Gordon A. Welty
Wright State University
Dayton, OH 45435 USA

It has been claimed that the foundational questions about the Delphi technique of long-range forecasting have all been answered. However, two unanswered foundational questions are considered in this paper. As a problem in the selection of experts, empirical evidence is adduced showing that institutionally sanctioned expertise (e.g. the "professional") in long-range value forecasting is not cost-effective. Another problem is the possibility of an Asch-Sherif deception of experts, or deflection of expertise, in Delphic exercises. Empirical studies are cited where such deflection occurs. Implications of these factors for the use and misuse of the Delphi technique are then discussed. Note that we only discuss Delphi in value forecasting. First we will consider the problem of selection.

The Selection Problem in Delphic Exercises

As part of an ongoing study of the parameters which define the process of selecting experts for Delphic exercises, we have considered the effect of variation of levels and areas of expertise on long-range value forecasting. Both Helmer/1/ and Rescher/2/ have suggested that the Delphi technique is a viable one for the forecasting of values and their changes. Among the variations of expertise that we have studied are the prima facie variation of level and area of expertise. When we varied the level of expertise, we found no difference in Delphic forecasts of changes in American values by the year 2000 between college sophomore engineering students and RAND, IBM, and other "high-level scientists and science administrators." That finding caused us to ask "Is the natural scientist the best forecaster?" /3/ When we subsequently varied the area of expertise as well as the level of expertise, we again found no difference between college sociology majors and the scientists in Delphic forecasts of changes in American values by the year 2000. /4/

While it appears that prima facie variation in levels and areas of expertise do not contribute to differences in forecasts of values and value changes, it may be otherwise for variation in expertise which is institutionally sanctioned. One of the major institutions for the sanctioning or "licensing" of expertise is the profession. /5/ While there is some disagreement in the social science literature on the defining criteria for a profession, all definitions include the notion of expertise and its restriction or licensing by the group./6/ It is of interest to consider to what extent we could suppose that professional status would constitute a criterion for the selection of forecasters for long-range value forecasting exercises. We shall find in the literature that conditions are assumed to obtain which are sufficient for the professional to be superior to the layman in long-range value forecasting. If these conditions do in fact obtain, and are indeed sufficient conditions, the problem of selecting value forecasters in Delphic exercises will be considerably alleviated.

Professor Hughes has emphasized the normative orientation of the professions. Of professions, he maintains "collectively they presume to tell society what is good and right for it in a broad and crucial aspect of life." Not only do the professions have a normative orientation; additionally, they have a normative function. He continues that professions "set the very terms of thinking" about what is right and good./7/ Thus a necessary condition for the professional to be a better than lay value forecaster is assumed, in the social science literature. The professional has an intimate and profound concern with valuational issues; he prescribes, evaluates, judges, praises, condemns, and above all, he professes.

The professional's assessment of the world is based upon what Hughes calls "the license to think relatively about dear things and absolute values." /8/ As a consequence of this relativity, we would expect the professional to have an assessment which differs from that of the layman. Hughes confirms this. "One order of relativity has to do with time; the professional may see the present in longer perspective. The present may be for him, more crucial in that it is seen as a link in a causative chain of events...In another sense, it appears less crucial, since the professional sees the present situation in comparison with others; it is not unique..." /9/

Hence a second necessary condition for the superiority of the professional over the lay value forecaster is asserted in the literature; the professional has a more comprehensive time horizon than the layman. Jointly, the two conditions are sufficient for the following proposition. The professional will be superior to the layman in the long-range forecasting of values. /10/ It is this proposition that we will now empirically examine, for the case of the marketing profession.

Marketing is not a profession of the stature of law and medicine. Nonetheless it appears to meet the criteria of a profession: there is a systematically organized corpus of knowledge germane to marketing, which is called "marketing theory;" /11/ there is a patent client orientation, and there is evidence of a concern for ethical questions pertaining to the behavior of the professional marketer. /12/

Finally, and of great importance for our interests, the marketing profession has a distinct interest in long-range forecasting, /13/ including the forecasting of values and their changes./14/ This stands in marked distinction from other professions, where the notion of forecasting is virtually unknown. The legal profession, for instance, because of the nature of the "legal method" and the case-study approach which underlies legal education, evidences neither interest in, nor understanding of, forecasting. If the legal profession had been selected as the locus of institutionally sanctioned expertise, one would be forced to restrict the subject matter of forecasts to predictions of judges' decisions, clearly short-run forecasts./15/

We will now describe an attempt to study the problem of selection of experts for Delphic exercises. We have replicated a substantial portion of Professor Rescher's research into the future of American values./16/ Essentially, Rescher's generic question consists of thirty-seven items each of which identifies an American value; an example is "success". For each of the items, representing a value of American society in the year 2000, an opinion of the probable change in emphasis of this value was elicited on a five point scale (ranging from 1 = greatly increased emphasis to 5 = greatly decreased emphasis).

The two groups compared consisted of professional American Marketing Association members (hereafter "AMA") and laymen, who were first-year graduate students in an MBA program who were not AMA members (hereafter "Students"). Thus the area of expertise was held relatively constant, and level of institutionally sanctioned expertise was varied. Specifically, a one-third random sample of the membership list of the Washington, D.C. chapter of the American Marketing Association provided 100 members of AMA who were mailed questionnaires including the 37 items, a cover letter explaining the Delphic exercise, and a stamped return envelope./17/ There were 59 respondents. Descriptively, 34% of the AMA respondents had a bachelor's degree and 44% had a master's degree. 47% of the respondents were under 35 years of age, 22% were between 36 and 50, and the remainder were over 51 years old. 40% of the respondents reported 10 years or less experience in the marketing field, 30% reported 11 to 20 years experience, and 30% reported over 20 years experience. A first year graduate class in the Department of Marketing, School of Business Administration, American University was the source of 20 student respondents, who were directly administered the questionnaire./18/ While all the AMA and student respondents received information on the Delphic exercise, and instructions in filling out the questionnaire, none was apprised of the comparative research involved.

The responses were tallied and the response patterns of the two groups were compared by the F-test. The overall multivariate F with 77 degrees of freedom was not significant at p = .05. The posterior pairwise analysis of the responses of the two groups on the 37 items by the t-test shows that only for two items (namely Item #1 and Item #3) were the means different for the AMA group and the student group, at p = .05. (See Table I following where the 37 item mean and standard deviations for the two groups are listed).

[Note: The items here coincide with those listed in N. Rescher "A Questionnaire Study of
American Values by 2000 A.D." K. Baier and N. Rescher (eds.) Values and the Future,
New York: Free Press (1969), p. 145.]

Thus it appears unlikely that variation in levels of institutionally sanctioned expertise makes a difference in long range value forecasts. The two item means where differences between the groups were observed, addressing the values of "one's own pleasure" and "economic security" respectively, scarcely evidence the relevance of expertise to the forecasting of values and their changes. It follows that expertise in long-range value forecasting by the Delphi technique is not cost-effective, since the laymen forecast the same as the professionals, and (presumably) cost less in terms of honoraria, etc.

So much for the selection of experts. Let us now turn to the deflection of expertise in Delphic value-forecasting exercises.

The Deflection Problem in Delphic Exercises

An element of the rationale for the Delphi technique is the supposition that social psychological influence processes present in the conventional committee are eliminated in the Delphi exercise. /19/ While face-to-face influence pressures such as rhetorical skill are clearly absent in nominal groups, it is entirely possible that other social psychological influence processes operate in the Delphi exercise.

As Allport argued in the early twenties, "to think and to iudge with others is to submit one's self unconsciously to their standards."/20/ Specifically, in the absence of objective measures and standards, opinions and judgements of individuals are more or less susceptible to conformity pressures towards group norms. This proposition was experimentally established by Sherif in the mid-thirties;/21/ the best known research in this area of conformity research is that of Asch./22/ In an Asch-Sherif type conformity study, the subjects, members of a larger group composed of the experimenter's colleagues ("shils"), are deceived by the experimenter and shils as to the nature or value of the group norm or consensus. The extent to which the subjects' judgement varies subsequent to this deception is the measure of conformity tendencies./23/ Major findings of Asch and subsequent researchers in the area include (a) an isolated subject is likely to exhibit conformity tendencies, depending upon personality characteristics such as self-esteem, which serve to sufficiently internalize norms of independence to counteract the effect of the group norm, or consensus /24/ and (b) several subjects are not likely to exhibit conformity tendencies./25/ When two or more subjects are present in an Asch-Sherif type study, they form a group of their own with true norms of its own, in opposition to the shils and the pretended norms or consensus of their group.

To what extent might such an influence process operate in the Delphi exercise? Clearly objective measures are absent in much of the forecasting subject matter upon which the Delphi technique is applied. It is precisely the absence of well-defined measures, time-series, etc. which requires judgmental forecasting. Further, the experts who participate in the Delphic exercise are "isolates", to guarantee their anonymity and to prevent the operation of face-to-face social psychological influence processes. Thus the conditions of the Sherif proposition on conformity behavior are fulfilled. Only two questions remain. Is "expertise" a sufficiently internalized norm to counteract the effect of the group norm or consensus?/26/ If not, the possibility always exists that the manager of the Delphi exercise can manipulate and distort the experts' opinions and judgement. The second question asks whether "expertise" might not be a norm which enhances the conformity inducing effect of the group norm. Let us look at the first question.

In Delphic exercise to forecast values for higher education in Virginia, Cyphert and Gant provided a provisional answer to this question./27/ Emphasis on sixty-two values or goals of the School of Education of the University of Virginia, including such items as "Increasing the number of talented young people who enter and remain in the teaching profession," were forecast for a ten-year period. There were seven groups of Delphi participants, including politicians, educators, etc.

In the first round of the exercise, a bogus item reading "Emphasizing the production of doctoral candidates in education who can improve the programs in schools of medicine, law, nursing and engineering" was introduced to the Delphi questionnaire and was rated low. The authors arbitrarily modified the rating that this item received; in addition they prepared fictitious rationales for its new pretended rating. By the final round of the exercise, the bogus item "was rated considerably above average, although not among the ten highest ranked targets. The hypothesis that the technique can be used to mold opinion as well as to collect it was supported."/28/

Hence, subsequent to deception, the subjects' judgement varied. Conformity tendencies were exhibited, tendencies which would not have appeared but for the enforced anonymity of the Delphi technique. Cyphert and Gant did not measure differential conformity tendencies across the seven groups, but in a personal communication, Dean Cyphert indicated that every group exhibited variation in its judgement subsequent to the deception. This suggests that (a) expertise cannot counteract social psychological influence pressures (whether or not expertise was variable across the authors' several groups), and (b) influence processes do remain operative within the Delphic exercise.

In subsequent research motivated by all the foregoing considerations, we attempted another test of the potential for influence processes within the Delphic exercise./29/ The experimental stimulus (the deception) was somewhat weaker than that presented to Cyphert and Gant's groups./30/ Our findings were equivocal but suggestive of the need for further research. Such factors as the magnitude of extremeness, the deception, the operation of cognitive vs. normative influence, etc. must be explicitly considered in subsequent studies./31/ Our subjects were students; we will examine that factor here.

Perhaps an explanation of the difference in findings here can be provided by reference to a very interesting passage in Bryce. He distinguishes opinion leaders (our professionals or "experts") from the "average citizen" (our laymen). Because of his expertise, the opinion of the expert has more "individuality, as well as more intellectual value. "/32/ But the expert is thereby far more autonomous. Nietzsche, for instance, distinguishes "the gregarious" and "the solitary" as a dimension of personality independent of such social dimensions as degree of expertise./33/

Bryce continues "he has also a stronger motive than the average citizen for keeping in agreement with his friends and his party, because if he stands aloof and advocates a view of his own, he may lose his influence and his position."/34/ This refers explicitly to the political arena, but "party" could probably be replaced by "invisible college" without straining Bryce's argument. Presumably the Delphic mechanism shields the expert from this sort of pressure. But Bryce goes on to introduce the notion of the expert's self-conception for which there is no shield.

Thus there are differential factors which may make the expert more susceptible than laymen to influence formation a la Asch. Bryce concludes that the opinion leader "is accordingly driven to make the same sort of compromise between his individual tendencies and the general tendency which the average citizen makes," /35/ and supposes that "'he makes it more consciously,'' which could enhance or dampen an Asch-type effect./36/

Surely this is an empirical problem. Cyphert and Gant had several groups in their exercise, of varying expertise. The students were comparable to their least expert group. Their observed Asch-effect may have occurred in the more expert group. In the absence of the empirical research to settle the question of the discrepancy between Cyphert and Gant's findings and the findings reported here, we will have to be satisfied with this resolution, tentative as it is. Only the estimation of between groups variance by Cyphert and Gant, and the testing of an explicitly directional hypothesis that a greater shift is positively related to greater "expertise", can definitively resolve the issue.

Certainly the presence of social psychological influence processes in the Delphic exercise is a disturbing problem. In politically sensitive forecasting subject matter, motivation exists for the manager of the exercise deliberately to distort the experts' opinions. Of course, "expertise", manipulated or not, is its own source of legitimacy./37/

Thus we can conclude this paper by stating that we find the selection of experts for the Delphic exercise, and the possibility of deflection of expertise within the Delphic exercise still to be problems worthy of further consideration. Hence we cannot agree with anyone who would suggest that the foundational questions about the Delphi technique have all been answered.

1. Olaf Helmer, "The Delphi Technique and Educational Innovation", Werner Hirsch et al (eds), Inventing Education for the Future, San Francisco: Chandler (1967), p. 81.

2. Nicholas Rescher, "Delphi and Values," P-4182, Santa Monica: RAND Corporation (September 1969), p. 5.

3. Gordon Welty, "A Critique of Some Long-Range Forecasting Developments," Bulletin of the International Statistical Institute, Vol. XLIV: 2 (1971), p. 405.

4. Gordon Welty "Problems of Selecting Experts for Delphi Exercises," Academy of Management Journal, Vol.15:1 (1972), p. 123.

5. Everett C. Hughes, Men and Their Work, Glencoe: Free Press (1958), p. 78; Burkart Holzner, Reality Construction in Society, Cambridge: Schenkman (1972), Chap. 9.

6. Cf. P. L. Stewart and M. G. Cantor, "Social Context of Occupations," P. L. Stewart and M. G. Cantor (eds.) Varieties of Work Experience, Cambridge: Schenkman (1972), Chap. 2.

7. Everett C. Hughes "The Study of Occupations" R. K. Merton et al, Sociology Today, New York: Basic Books (1959), p. 447. D. L. Currill has noted "the extremely careful selection of Delphi participants required, where it is of importance not to mistake interest for knowledge." Cf. his "Technological Forecasting in Six Major U.K. Companies," Long Range Planning, Vol. 5:1 (1972) p. 74. Given this consideration in the selection process, the professional would appear to be very likely to represent knowledge as well as interest.

8. Ibid. p. 449.

9. Ibid. p. 450.

10. Sufficient conditions for this proposition can be found as early as Hegel. "Der offentliche Stand...haben...erfulltes Denken, die Intelligenz, die sich selbst weiss." G. W. F. Hegel Jenaer Realphilosophie (1805) Hamburg: Felix Meiner (1967), pp. 259-260.

11. An early statement is W. Alderson and R. Cox "Towards a Theory of Marketing," Journal of Marketing, Vol. 13 (1948), pp. 137-152.

12. Cf. Robert Bartels "A Model for Ethics in Marketing" Journal of Marketing, Vol. 31:1 (1967), pp. 20-26.

13. Cf. W. Alderson "Forecasting Structural Change in Long Range Planning," Robert Kaplan (ed.) The Marketing Concept in Action, Chicago: American Marketing Association (1964), pp. 146-153; Maxwell Hunter "Are Technological Upheavals Inevitable?" Harvard Business Review, Vol. 47:5 (1969), p. 77

14. John Howard, Marketing Management, Homewood, IL: Irwin (1963), p. 138; James Bright, "Evaluating Signals of Technological Change," Harvard Business Review, Vol. 48:1 (1970), p. 64.

15. Cf. Oliver Holmes "The Path of the Law," C. Morris (ed.) Great Legal Philosophers, Philadelphia: University of Pennsylvania Press (1959), pp. 421-434; Julius Stone, Social Dimensions of Law and Justice, Stanford: Stanford University Press (1966), pp. 62-71.

16. Nicholas Rescher, "A Questionnaire Study of American Values by 2000 A.D." Kurt Baier and N. Rescher (eds.) Values and the Future, New York: Free Press (1969), pp. 140-145. We would like to acknowledge the assistance of Mr. Harold Levy in this research.

17. We would like to thank Mr. Walt Winnard of AMA for his assistance here.

18. We would like to thank Mr. Robert Baughman for his assistance in administering these questionnaires.

19. Frederick R. Cyphert and Walter L. Gant, "The Delphi Technique," Journal of Teacher Education. Vol. XXI:3 (1970), p. 418.

20. Floyd H. Allport, Social Psychology, Boston: Houghton Mifflin (1924), p. 278.

21. Muzafer Sherif and Carolyn W. Sherif, Social Psychology, New York: Harper and Row (1969), Chapter 10, esp. pp. 206-207.

22. Vernon Allen, "Situational Factors in Conformity," in Leonard Berkowitz (ed.), Advances in Experimental Social Psychology, Vol. II, New York: Academic Press (1965), pp. 133-175, esp. p. 134.

23. Charles Kiesler and S. B. Kiesler, Conformity, Reading: Addison-Wesley (1969), pp. 2-3.

24. Arthur Cohen, "Sone Implications of Self-Esteem for Social Situations," Carl Hovland and I. L. Janis (eds.), Persuasion and Persuasibility, New Haven: Yale University Press (1959), pp. 102-120.

25. Solomon Asch, Socia1 Psychology, Englewood Cliffs: Prentice-Hall (1952), pp. 477-479.

26. Wm. Chambliss and Robert Seidman, Law, Order and Power, Reading: Addison Wesley (1971), p. 148 suggest this is the case for appellate court judges.

27. Cyphert and Gant, op.cit., pp. 417-425.

28. Cyphert and Gant, op.cit., p. 422.

29. Gordon Welty, "A Critique of the Delphi Technique," Proceedings of the American Statistical Association (Social Statistics Section), Washington, D.C. (1972), pp. 377-382.

30. The relevance of group attractiveness is discussed in Allen, op.cit., pp. 149-152.

31. Cf. Allen, op.cit. pp. 155-157 and 162-164.

32. James Bryce, The American Commonwealth, Vol. II., New York: Macmillan (1891), p. 245.

33. Frederick Nietzsche, The Will to Power, Random House (1967), pp. 472-473.

34. Bryce, op.cit., p. 245.

35. Bryce, Ibid.

36. Bryce, Ibid.

37. Holzner, Reality Construction in Society, op.cit. pp. 133-134.