"The Internal Validity of the Kuhn-McPartland TST," Proceedings of the American Statistical Association (Social Statistics Section), Washington: ASA (1972), pp. 190-193.

by Ronnie M. Alm, Walt F. Carroll and Gordon A. Welty

[//190] Introduction
The sociological theory called symbolic interactionism derives from the writings of George Herbert Mead. It is a theory of the development of the self from one's social milieu, and from perceptions of and interaction with others. (See, e.g. Blumer [1] for a succinct account of the Meadian theory.)

It has been charged that Mead, and by implication his followers, eschewed empirical research ([6], pp. 238-l39). This is presumably to the detriment of symbolic interactionist theory. While this charge is unquestionably true of Mead, it is not an accurate assessment of present day symbolic interactionists.

In an attempt to operationalize one of the key concepts of symbolic interactionist theory, that elusive concept of the "self," and to further empirical research on the self-concept, Manford Kuhn developed the Twenty Statements Test [4]. Kuhn and his collaborator McPartland supposed that human behavior is organized, the organization supplied by the individual's self-attitudes or self-concept. Following the direction provided by Mead's insistence on language as the basis of socialization and the self, it was straightforward to propose that self-attitudes could be assessed by asking an individual verbally to characterize himself.

The test (hereafter TST) is a sheet of paper, at the top of which are the instructions: "There are twenty numbered blanks on the page below. Please write twenty answers to the question 'Who am I?' in the blanks. Write the answers in the order they occur to you; don't worry about logic or importance." Up to twenty responses to this generic question provided subjective definitions of the self for Kuhn, which he understood as internalizations of a person's objective social status. Hence the organization of behavior could be analyzed in terms of the self-conception, which Kuhn (and Mead) understood as a plan for behavior. Thus the subjective, and perhaps more importantly, the anticipatory aspect of the self, based in part upon ex ante data configurations permitted by symbolization, could be compared with the objective aspect of behavior, known only in terms of ex post data.

The test has the virtues of simplicity and of providing a relatively direct measure of one's self-concept. For such reasons among others, the TST has been utilized in a number of social scientific studies (cf. [7] for citations).

Two Major Methodological Problems
However, the substantive interest in the TST has not been matched by a concern for several striking methodological problems that bear on the test as an instrument to assess the self-concept. There has been one major study of the TST which was properly methodological in focus. This was Tucker's examination of Kuhn's theory of the self and the TST as an instrumentality for operationalizing this theory. After characterizing Kuhn's conception of "Self" as a theoretic entity, Tucker notes that it followed for Kuhn that "self-attitudes are derived within a particular 'context of behavior' and are 'meaningless' without the explication of that 'context'." ([12], p. 312)

Kuhn's inference would appear to be wholly compatible with Mead's own thoughts on the self and the self concept. Particularly, Mead notes the relevance and importance of other socially proximate symbol-using actors to the development of the self and one's conception of his self (cf [5], pp. 160-161).

Unfortunately for systematic empirical research, however, such an implication is at variance with one requirement that must obtain for any test. Tucker points out another implication of the use of the TST in operationalizing social psychological theory is that "it is assumed that the responses from the question, 'Who am I?', are applicable to a variety of situations. This assumes that the others who are 'present' and 'contemporary' are irrelevant to the person's behavior" ([12], p. 312).

These two implications are clearly incompatible. Of particular concern is the relevance to the test subject of the presence of the researcher employing the TST. Either the responses will reflect the idiosyncracies of a "context of behavior" which for our purposes happens to include a "researcher" and is characterized by the label "social scientific research," and the first implication Tucker draws from the self-theory is affirmed and the second denied. Alternatively, the responses will not exhibit variation due to the subject's sensitivity towards a unique social situation. Since in most social contexts there is no researcher present, if the presence of the researcher administering the TST is without effect, the second implication drawn by Tucker will be affirmed, and the first denied.

Another major problem is the incompatibility with theoretical premises of the self theory of the "content analysis procedures" or coding techniques used in categorizing the TST responses. Tucker notes that three basic assumptions of self theory are the slight predictive utility of fixed responses since the self is a function of the social context; that the Twenty Statement Test gives a valid representation of the subjects' self concept; and that the focus of the research is explicitly perspectival ([12], p. 313).

A methodological problem arises if, when coding the TST responses, a coder other than the subject [190/191] "imposes the meaning of each of [the responses] from his own perspective" ([12], p. 3l3). Obviously, this is incompatible with the basic premises of self-theory.

If direct acquaintance with the self under study is required for the valid categorization of responses then the basic assumptions that Tucker notes of self theory are affirmed; thus the assumption that a coder other than the subject can impose his own meanings on the responses is denied. Otherwise, the responses can be considered to be relatively fixed, hence invariant to the identity of the coder. In the latter case, the second condition noted by Tucker will be affirmed, and the premises of self theory denied.

Problem of Situationality.
When an instrument such as the TST is employed in measuring the self-concept, it is possible that the presence of the researcher may result in different responses from the subjects than the responses that would be occasioned were the researcher absent. The outcome of differential effects of the presence of the researcher requires differential response to the research situation itself, rather than the substantive variations of independent variables, induced by the researcher, within the research situation. One way that differential response to the research situation can be ascertained is by measuring the willingness on the part of the subjects to volunteer for a social scientific study.

It has been documented that there is a volunteer effect, where the subject's sensitivity increases to the "cues" that the experimenter unconsciously provides as to the hypothesis being tested, as motivation to participate as evidenced by volunteering increases (cf. [9] for references and discussion). Professor Orne calls these cues the "demand characteristics" of the experiment, i.e., that which the experimenter (not necessarily consciously) demands of the subject in terms of the substantive variables of the experiment. Orne has been quite explicit: "Within the context of our culture the roles of subject and experimenter are well understood and carry with them well defined mutual role expectations" ([8], p. 777).

Not only is the volunteer subject more sensitive to these cues, but in terms of his perceptions of the cues, he strives (again, not necessarily consciously} to "please" the experimenter by helping realize the prediction, substantiate the hypothesis, etc., by his own behavior. The generic hypothesis we sought to test was that the volunteer effect results in variations in the emerging self-concept measured by the TST.

The subjects were 56 undergraduates enrolled in an introductory sociology course. The volunteer/ non-volunteer status of these subjects were ascertained by a procedure similar to that reported by Rosnow and Suls [10]. Immediately subsequent to the administration of the TST, a colleague of the investigator entered the classroom and solicited volunteers for a fictitious experiment in psycholinguistic research to take place several months later. He told the subjects that he would pay each of them $1.00 for a half-hour's participation. Participation in the (fictitious) experiment could be arranged for times mutually convenient to the pretended researcher and the subject. Following Rosnow's rationale, the financial incentive was offered to lend credibility to the experiment, yet was not large enough to provide a motive for participation. Sixteen of the subjects volunteered to participate. At the same time as he took the names and addresses of these volunteer subjects, the pretended researcher collected the anonymously completed TST sheets. Forty of the subjects did not volunteer. Since the subjects were unaware of the hypotheses being tested, they were essentially participating in a single-blind experiment.

There are two major schemes for coding the responses to the TST. The first is the compilation of the subjects' locus score, which has been taken as a measure of the extent to which the subject is anchored in the social system ([11], p. 50). The second is the categorization of the responses into a set of five analytical categories developed by Kuhn. We have utilized the second scheme in this investigation.

Full definitions of the five categories are given in Schwirian's essay ([11], p. 51). We merely list the categories with brief explication. They include (1) Consensual responses, i.e. statuses in social categories and social groups, (2) Ideological beliefs, i.e. religious and philosophical orientations, etc. (3) Aspirations, i.e. future-tensed statements of personal goals and achievements, (4) Preferences, i.e., the respondent's interests and aversions, and (5) Self evaluation, i.e. evaluative statements assessing one's own mental and physical abilities and demeanor. The 56 TST sheets were coded in terms of these five categories.

Six specific hypotheses were generated. A null hypothesis stated that there would be no significant difference between volunteer and non-volunteer subjects in the mean number of responses for each of the five categories listed above. In addition to these five null hypotheses, one null hypothesis stated that there would be no significant difference between volunteer and non-volunteer subjects in the total number of responses given. It was further supposed that this sixth hypothesis was directional. Volunteer subjects would strive more diligently than non-volunteer subjects to comply with the instructions to "write twenty answers to the question 'Who Am I?'"

The data is summarized in the following table where the mean number of responses per analytical category (and associated hypothesis) as well as the standard deviation of responses per category is given for the two groups. Student's t was utilized to test the differences in means. Data on the significances of the observed differences is given in the right hand column.

 

Response Category

Non-volunteers (n = 40)

Volunteers (n = 16)

Significance of Student's t

 

 

Mean

s

Mean

s

 

1.

Consensual

6.70

5.12

6.19

4.00

n.s.

2.

Beliefs

0.93

1.25

2.25

2.25

p = .01

3.

Aspirations

0.25

0.62

0.81

1.07

p = .05

4.

Preferences

1.68

1.68

2.94

2.84

p = .05

5.

Evaluations

6.15

5.05

5.88

4.62

n.s.

6.

[Omissions]

4.40

4.97

1.94

3.70

p = .05

 

[191/192] The following null hypotheses were sustained: (1) There is no significant difference between the volunteer and non-volunteer subjects, in the mean number of consensual responses given, and (5) there is no significant difference between the volunteer and non-volunteer subjects, in the mean number of evaluative responses given.

The other four null hypotheses were not sustained, and we retained the following research hypotheses: (2) There are significantly more ideological belief responses given by volunteer than by non-volunteer subjects, (3) there are significantly more aspiration responses given by volunteer than by non-volunteer subjects, (4) there are significantly more preference responses given by volunteer than by non-volunteer subjects, and (6) there are significantly fewer omissions given by volunteer than by non-volunteer subjects.

The Problem of Coding
Another question arises from Tucker's discussion that bears on the methodological problems of the TST, in considerations of theoretical constructs on which the instrument is based, versus the content analytical (or coding) procedures ([12], p. 303).

Tucker pointed out and Franklin and Kohout have noted, at the time of analysis, the researcher imposes his own meanings on the subjects' statements by coding them according to a set of "a priori" categories ([2], p.82).

The question then arises, "Must the subject score his own test for test validity?" This question would appear to be methodologically germane to any explicitly perspectival test. Franklin and Kohout compared the locus scores (consensual statements) calculated on the basis of the subject's own coding and those calculated from the researchers' codings. These results were obtained by having the subjects and the researchers code the statements, with the "other" coders who were the researchers scoring the tests independently and later resolving inter-coder discrepancies. Franklin and Kohout's data indicated that the empirical consequences of having the TST responses coded by using the "self" versus the "other" codings "are probably not significantly different" ([2], p. 88).

The present research likewise focuses on the possible discrepancies between "self" and "other", coding. We replicated the Franklin and Kohout test circumstances. However, we have categorized the statements into the set of five analytical categories developed by Kuhn, ([4]) instead of the two broad categories of consensual and sub-consensual statements.

The subjects were thirty undergraduates enrolled in an introductory sociology course. After completing the TST in a 12-minute period, the subjects were given coding instructions based on Kuhn and McPartland's definitions of each of the five categories ([4], pp. 40-41). The "other" coders were researchers familiar with both the TST and its analytical procedures and with self-theory. The researchers also coded the test independently, later resolving inter-coder discrepencies as specified by the Franklin and Kohout coding procedure ([2], p. 85). In addition, an average of the other coder analyses was made. This coding procedure was introduced to ascertain if a linear composite of coding outcomes was equivalent to the self coding, thus providing an alternative to the Franklin and Kohout coding procedure. First we will discuss the comparison involving the Franklin and Kol1out procedure.

For each of the five coding categories, there was one null hypothesis developed, each one stating there would be no significant difference between the mean number of responses for that category when coded by the subject himself, and when derived from the Franklin and Kohout resolution of coding behavior of several researchers.

 

Response Category

Self-Coding

Other

 

 

Mean

s

Mean

s

1.

Consensual

5.227

2.861

5.136

3.299

2.

Beliefs

1.364

1.965

1.409

1.501

3.

Aspirations

1.727

1.162

0.364

0.581

4.

Preferences

1.409

1.563

0.909

1.444

5.

Evaluations

10.273

4.600

10.909

4.524

 

All the null hypotheses were sustained; the test of significance was Student's t at p = .05.

We will now discuss the comparison of self coding with the linear composite of the coding behavior of the several researchers.

For each of the five coding categories, again there was one null hypothesis developed, each one stating there would be no significant difference between the mean number of responses for that category when coded by the subject himself, and when derived by averaging the coding behavior of the several researchers.

 

 

Response Category

Self-Coding

Other

 

 

Mean

s

Mean

s

1.

Consensual

5.227

2.861

5.091

3.366

2.

Beliefs

1.364

1.965

1.485

1.352

3.

Aspirations

1.727

1.162

0.440

0.548

4.

Preferences

1.409

1.563

1.152

1.093

5.

Evaluations

10.273

4.600

10.380

4.372

 

All the null hypothesis were sustained; again, the test of significance was Student's t at p = .05.

Thus our results to this point corroborate the earlier findings of Franklin and Kohout that the score is invariant under other coding procedures operationally equivalent to subject coding methods, either when inter-coder discrepancies were resolved as specified in the Franklin and Kohout procedure, or when the "self" coders were compared with the average of the "other" coders ([2], p. 82).

However, both Franklin and Kohout procedure, and the procedure of linear composites might significantly and artificially suppress inter-coder variance. Thus, we developed five null hypotheses, each one stating that there would be no significant difference between the mean number of responses as coded by each of the researchers for that category. This data is summarized, in the following table. [192/193]

 

Response Category

Coder #1

Coder #2

Coder #3

Significance of Fisher's F

 

 

Mean

s

Mean

s

Mean

s

 

1.

Consensual

4.682

3.414

5.773

3.664

4.818

3.290

n.s.

2.

Beliefs

1.727

2.074

0.455

0.671

2.273

2.142

p =.05

3.

Aspirations

0.409

0.590

0.682

0.839

0.227

0.528

n.s

4.

Preferences

0.636

1.217

2.318

1.810

0.500

0.964

p = .05

5.

Evaluations

11.273

4.682

9.500

4.296

10.364

4.924

n.s

 

The test of significance was Fisher's F at p = .05.

We found significant differences between the "other" coders in two of the five categories (the ideological and interest categories) and an almost significant difference in the category of aspirations. This suggests that the resolution of coding disagreements by either the Franklin and Kohout coding procedure or the linear composite procedure arbitrarily suppresses significant variance. This finding was corroborated by a pairwise posterior analysis of the five categories for the self coding and the coding behavior of each of the researchers. Again the test of significance was Student's t at p = .05, and for two of the researchers, significant differences were found between their coding behavior and self coding in one category each.

Thus we conclude that in the absence of further research into coder differences one cannot take any particular "other" coder to code the TST since there does appear to be an individual coder effect.

In conclusion, there are several propositions that appear warranted. First, there are significant volunteer effects on the responses to the TST. Thus, the responses to the TST do not appear to be unconditionally applicable to a variety of situations. Second, the Franklin and Kohout coding procedures appear to be operationally equivalent to self coding of the TST. Finally, significant inter-coder variation appears to exist, both between research coders themselves, and between them and the subjects as coders.

References

(1) Blumer, Herbert, "Society as Symbolic Interaction," in Rose, A. Human Behavior and Social Processes, Boston; Houghton Mifflin (1962), pp. 181-184.

(2) Franklin, Billy J. and Kohout, Frank J., "Subject-Coded Versus Researcher-Coded TST Protocols: Some Methodological Implications," Sociological Quarterly, Vol. XII (Winter,1971), pp. 82-89.

(3) Hickman, C.A. and Kuhn, Manford, Individuals, Groups, and Economic Behavior, New York: Dryden Press (1956).

(4) Kuhn, Manford and McPartland, T.S. "An Empirical Investigation of Self-Attitudes," American Sociological Review, Vol. 19 (1951), pp. 68-76.

(5) Mead, George Herbert "A Behavioristic Account of the Significant Symbol," Journa1 of Philosophy, Vol. 19 (1922), pp. 157-163.

(6) Merton, Robert K. Social Theory and Social Structure, Glencoe: Free Press (1957)

(7) Nass, Gilbert. "Literature on Measurement of the Self-Concept," Journal of Counseling Psychology, Vol. 8 (1961), p. 368.

(8) Orne, Martin. "On the Social Psychology of the Psychological Experiment," American Psychologist, Vol. 17 (1962), pp. 776-783

(9) Rosenthal, Robert and Rosnow, Ralph "The Volunteer Subject," in Artifact in Behavioral Research, ed. R. Rosenthal and R. Rosnow, New York: Academic Press (1969), pp. 59-118.

(10) Rosnow, Ralph and Suls, J. "Reactive Effects of Pretesting in Attitude Research," Journal of Personality and Social Psychology, Vo1. 15 (1970), pp. 338-343.

(11) Schwirian, Kent, "Variations in Structure of the Kuhn-McPartland Twenty Statements and Related Response Differences," Sociological Quarterly, Vol. 5 (1964), pp. 47-59.

(12) Tucker, Charles, "Some Methodological Problems in Kuhn's Self Theory," reprinted in Manis, Jerome G. and Meltzer, Bernard N. Symbolic Interaction: A Reader in Social Psychology, 2nd ed. Boston: Allyn and Bacon, 1972, pp. 303-15.

[193//]