SOCIAL SCIENCE RESEARCH COUNCIL
VOLUME 32 • NUMBERS 3/4 • DECEMBER 1978 605 THIRD AVENUE • NEW YORK, N.Y. 10016
Herbert A. Simon Awarded 1978 Nobel Memorial Prize in Economics: Council Board Member, 1958-71 HERBERT A. SIMON, who was awarded the 1978 Alfred Nobel Memorial Prize in Economic Science in Oslo on December 10, has had a long association with the Council, and the announcement on October 16 of his selection was particularly gratifying to Council members. A political scientist who serves as Richard King Mellon university professor of computer science and psychology at Carnegie-Mellon University, Mr. Simon has exemplified in his distinguished career the interdisciplinary purposes and program of the Council. In making the award, the Royal Swedish Academy of Sciences cited Mr. Simon's "pioneering research into the decision-making process within economic organizations," adding that "modern business economics and administrative research are largely based on Simon's ideas." A member-at-large of the Council's board of directors from 1958 until 1971, Mr. Simon was board chairman from 1961-65. He was a member of the Committee on Problems and Policy from 1962 through 1968; here his multidisciplinary interests and his rigorous standards for research made him an invaluable member. He was a member of the Committee on Business Enterprise Research from 1953 to 1957 and of the Committee on the Simulation of Cognitive Processes from 1957-62; he was chairman from 1959-62. In the summer of 1958, Mr. Simon and Allen Newell codirected a now-historic four-week
Contents of this issue-see page 4J 41
summer trammg institute in computer simulation The Council has benefited enormously from Hertechniques. Held at the offices of the Rand Corpora- bert Simon's wise counsel and his interest both in the tion, with funds provided by the Ford Foundation, simulation of decision-making processes and in the this workshop created a nucleus of trained faculty application of this knowledge to administrative and members on some dozen campuses. Much of the sub- business organizations. It is a pleasure to acknowlsequent research and publication in this field can be edge this debt on the occasion of his receiving the traced back to initial training received at this institute. 1978 Nobel Prize.
Indian Law as an Indigenous Conceptual System by Marc Galanter* When the Joint Committee on South Asia was reconstituted in 1976, it took as its major mandate the analysis of South Asian conceptual systems with a particular concern for their value in developing culturally sensitive analytical tools. The underlying notion was that many of the concepts of Western social science fit badly in non-Western societies, and efforts should be made to examine them in their own terms. In the September 1977 Items, Stanley J. Heginbotham, Congressional Research Service, who serves as chairman of the Joint Committee on South Asia, noted that the "committee'sfocus on South Asian perceptions of the world reflects a concern that social scientists, in formulating and utilizing analytic categories, logical linkages, and models, keep in mind that the people of the region often think in very different categories, use very different logical linkages, and assume distinctive models of human behavior." The committee is seeking to examine a number of different indigenous systems: notions of political authority, folklore, concepts of the person, the concept of karma, responses to. risk and uncertainty, and the legal system. The following reflections of Mr. Galanter on Indian law give an indication of the kinds of issues being addressed.
WHEN I first became interested in Indian law more than 20 years ago, I encountered some quizzical reactions on the ground that since most of Indian law is palpably Western in origin, studying it could not reveal much about India. Nearly ten years ago, I wrote a paper expressing my feeling of wonder that independent India had not seen any serious attempts to dislodge the legal system imposed on it by the British (Galanter 1972). Both intellectually and institutionally, there seemed an acceptance of the mid20th century system as something fully Indian. There is nothing, for example, like the conflicted but very genuine desire to get rid of the English language as a medium of public life. This 1972 paper tries to explain this lack of interest in dismantling the legal system; the findings boil down to a widespread feeling among professionals, the urban elite, and some villagers that "this is our system." 42
This situation points to the question of what we mean by an indigenous conceptual system. Is it one that the indigenes feel is theirs? Or do we have to look to some elite among the indigenes? If so, which elite do we look to? That is very much the question here, because in India there has been a displacement of an older elite, who were the carriers of an earlier indigenous tradition. This does not imply that the earlier legal complex was an exact counterpart of the law complex that developed during British rule, although each combined authoritative learning, governmental force, and local dispute institutions. 1 The strength of the modern law complex, I have argued, is that it introduced a new and powerful set of ties among these elements (Galanter 1968). The older legal tradition fit with other components of social life in ways recognizable to elites and to lay people alike-and to observers from afar. It elaborated such familiar notions as dharma and it was embodied in persons of religious and political eminence. What replaced it was an institutional-intellectual complex that does not fit neatly with the other institutions. In the new dispensation, there are discontinuities. But at the same time, law as an institution is more differentiated from everything else, so that its inconsistency with the internal content of other institutions (e.g., family, temples, castes) doesn't matter as much (cf. Mayhew 1971). Perhaps India has moved from a situation in which all the major intellectual-institutional complexes in the civilization were more or less compatible, based
* Marc Galanter. professor of law and South Asian studies at the University of Wisconsin. is a member of the Committee on Law and Social Science and the Joint Committee on South Asia of the Council and the American Council of Learned Societies. This paper was prepared for the October 1978 meeting of the Joint Committee on South Asia. I Rocher (1972) suggests that the British made a fundamental error in equating dharmasastra (Hindu systematic learning about duty) and its expositors with law and lawyers.
VOLUME
32,
NUMBERS
3/4
(in theory at least) on a common fund of ideas and worked out by a common fund of techniques. Now the institutional-intellectual complexes are bigger, more internally differentiated, more differentiated from each other, and more independent of one another. Thus, a complex like "law" can exist with its own internal socialization of key actors, its own technical vocabulary, its own styles of thought, its own prestige hierarchy, its own system of social support, and its own system of concepts and meanings. It forms a large and partly autonomous subculture of its own. Is this the case with other institutionalintellectual complexes as well (e.g., government, education, art, journalism)? When a society undergoes this kind of transformation, it seems to me that "indigenous" takes on a new meaning. In the old dispensation, one could spot a foreign element because of its lack of fit with others. But in the new world, institutions don't flow into and reflect each other in the same way. It is a world of discontinuities and abrupt shifts between different parts of one's life experience. The indigenes are not always recognizable or accessible to one another. There may be great barriers between one indigenous phenomenon and another. The law may loom as intrusive, alien, and external to many within Indian society, while others may find it comfortable, known, wholly familiar, and expressive of their values and outlook. Clearly, "indigenous" cannot mean just that which is not of foreign origin. How far back do you go? There have been many grafts on the tree. The Indians have taken British law and have reworked it, imposed unique stylistic features on it (Galanter 1967, 1968). The affinities with British or American law (there was a second wave of borrowing of American constitutionalism) are striking; the kinship is apparent, yet the Indian style is quite distinctive. As George Gadbois (1977) has pointed out, there must have been some special predilection among Indians underlying this extraordinary receptiveness. This eager and unfaltering embrace of Western law simply did not take place in Southeast Asia or in Japan (and I think this is true for Africa as well). The attachment to Anglo-Indian law (and its American-style constitutional overlay) runs deep. Two major new pieces of evidence suggest the tenacity of these forms in late 20th century India. First, there is the resistance to alterations in these institutional forms manifested in connection with the 1975-77 Emergency regime. Second, there is the disintegration of post-Independence efforts to revive nyaya panchayats (village tribunals). DECEMBER
1978
During the 1975-77 Emergency, there was a decisive break with the liberal constitutionalist vision of the legal order that was shared by the Bar, officials, the political elite, and a large part of the populace since before Independence-an order in which independent courts vindicated private rights and curtailed official arbitrariness. The major contours of the Emergency regime are familiar by now: the elimination of civil liberties, particularly by control of the press and by a reduction in the autonomy of organized groups such as labor unions and the Bar; the elevation of the executive to decisive supremacy over Parliament; and especially a reduction of the power of the .iudiciary to decide public issues. After a few gestures in the direction of repudiating the constitutional design, the government of Mrs. Gandhi pursued a policy of removing the courts from politically sensitive areas, making them more subserviant politically but retaining, thus truncated, the rule of law. The new constitutional amendments eliminated the judiciary's power to pass on the propriety of constitutional amendments, severely curtailed the power of higher courts to adjudge statutes unconstitutional, and eliminated the jurisdiction of the ordinary courts over such politically sensitive areas as government employment, revenue, land reforms, labor disputes, and elections. Roughly, all of the politically volatile cases were withdrawn from the regular courts and put into tribunals that would, presumably, have been more politically responsive. The supervisory power of the regular courts over these tripunals was curtailed or eliminated. But with these significant amputations-and subject to some attempts to influence the judges by arbitrary transfers, etc.-the regular courts were left to function as before. The pattern that emerged was reminiscent of what the
CONTENTS OF THIS ISSUE 41
Herbert A. Simon Awarded 1978 Nobel Memorial Prize in Economics 42 Indian Law as an Indigenous Conceptual SystemMarc Galanter 47 G~vernment Audits of Social Experiments-Ronald P. Abeles 51 Personnel -New directors and officers -Staff appointments 52 Recent Council Publications
43
Spanish sociologist Jose Toharia (1975, pages 494-5) suggests is the classical authoritarian response to law. Referring to the final years of the Franco regime, he wrote: The situation of the ordinary courts under the present Spanish regime [is characterized] ... on the one hand by the considerable degree of independence allowed to the judges (with a consequent lack of political indoctrination and the exercise among them of a certain ideological diversity) and on the other by the sharply curtailed and relatively unimportant sphere of action afforded them.
The administration of justice is thus controlled by avoiding the political mobilization of individuals and institutions and by "the systematic reduction of the area of competence of ordinary tribunals." In spite of these drastic incur~ions into the legal order, there were during the Emergency remarkable continuities. Most prominently, there was an adherence to superficial legal proprieties and an unwillingness to depart from the major institutional forms of Anglo-Indian law. Although there were a few official mutterings about people's courts, these did not amount to more than a scare. There seems to have been no serious consideration of replacing or transforming the familiar brand of Anglo-Indian courts either with "people's courts" or with indigenous panchayats. Indeed, the government undertook new commitments to support a massive legal aid program, presumably to increase access to and use of the legal system. 2 If there were outcroppings of a genuine desire to make legal institutions responsive to popular needs, there was little sense that these institutions would require alteration or transformation in order to accomplish this. Although both technocratic and populist themes were invoked during the Emergency to constrain the autonomy oflegal institutions, the new legal measures were in very much the same style that had previously animated courts, lawyers, and legislators. The Emergency measures invo~ved no attempt to reshape legal forms, but only sought to truncate and contain them. Law remained formalist rather than instrumentalist, a matter of individual entitlements conferred from the top rather than one of facilitating group mobilization, a matter for professionals rather than for lay participants. 2 While curtailing judicial power, the infamous 42nd Amendment to the Indian Constitution added a new Directive Principle that the State shall "provide free legal aid . . . to ensure that opportunities for securing justice are not denied to any citizen." The justice to be secured is, by implication, that dispensed by the then current variety of laws, COUTts, and lawyers.
44
As soon as the Emergency ended there was an immediate concern to restore legal institutions to their former condition. Is this restoration only a temporary reversal of a long run trend toward the extrusion of a foreign element? In that light, one might see Mrs. Gandhi as a precursor of a movement to develop truly Indian political and legal forms. But what is remarkable is that even during the Emergency there was no significant challenge to the basic form and style of legal institutions. There was griping from the government side about the courts interfering with what were thought to be issues that should be left to Parliament and the Executive. But the basic shapes and style of courts as institutions and of the legal learning that they cultivate and dispense was never questioned. Both Mrs. Gandhi and her opponents shared a basic acceptance of India's legal institutions. The second body of data is that concerning the disintegration of the nyaya panchayats (village tribunals) promoted by the government after Independence. In a recent assessment of this program (in press), Upendra Baxi and I found that the nyaya panchayats established in the 1950s and early 1960s are for the most part moribund. Where these tribunals have not been abolished by state governments dismayed at their workings, they have low case loads and enjoy little public or official regard. Although lack of government support undoubtedly made an important contribution to their demise, it is clear that they never attracted significant support from the villagers in whose name they were established. It is not clear whether they withered away because they lacked the qualities of the traditional indigenous tribunals or because they displayed them all too well. Most likely, it was because they represented an unappetizing combination of the formality of official law with the political malleability of village tribunals. Nevertheless, there now appears to be a new wave of interest in these institutions. Recent proposals to promote nyaya panchayats tend to visualize them as agents for the efficacious delivery of official justice rather than as means to revive indigenous justice. Thus, in a report (Government of India 1973) that views itself as a radical critique of Indian legal arrangements,Justice Krishna Iyer speaks glowingly of nyaya panchayats, citing Lenin in their favor, and commends them as a promising counterpart to "the system of justices of peace in the United Kingdom and People's Courts in the Soviet Union .... " These examples he reads as establishing that "laymen may well be entrusted with dispensing legal justice provided certain safeguards are written into the VOLUME
32,
NUMBERS
3/4
scheme."3 It is clear that the "legal justice" to be dis- comprised of a chairman, the panchayat judge, "havpensed is the law of the land and not that of the ing knowledge of law," and two respected lay memvillagers or their spiritual advisors. Nyaya panchayats bers who would be drawn from a select panel for each are commended as inexpensive, accessible, expedi- case by the panchayat judge. The lay members would tious, and suitable to preside over conciliatory hear- be given rudimentary legal training. The tribunal ings. The Report recommends that they be staffed by would proceed informally, would emphasize conciliasuperannuated judges and retired advocates. Pan- tion; there would be no professional lawyers, etc. chayat justice is seen as part of a larger scheme of legal Legal questions would be decided by the judge and aid and as providing public access to the courts. his decisions could be reviewed by the District Judge. Charged with proposing concrete measures to se- What seems to be proposed is an informal, conciliacure access to justice for the poor, the Bhagwati Re- tory, nonadversarial, small claims court with some lay port (Government of India 1978) elaborates this dis- participation. cussion of panchayats. After the obligatory nod to These reports indicate that even those Indian Mohandas Gandhi, the Report observes (in chapter 6) jurists who are most critical of the present system that don't visualize an "indigenous" alternative to it-at ... there can be little doubt that our legal and judicial system is least if we take "indigenous" to mean a return to not adequate to meet the needs of the new society which is earlier forms. I think it is fair to say that at the level of emerging in our country. It is not effective to provide a solution discourse about policy, there is an almost total obliterto the new problems which are coming up and presenting a ation of indigenous elements. 4 However, if one looks challenge to contemporary society. It is not sufficiently responsive to the new norms and values which are replacing the old at the social organization of modern Indian law, I and it does not reflect properly and adequately the new ap- think it is possible to identify elements that seem to be proach which characterizes the true purpose and function of the extension or projection of traditional patterns. law. For example, when I recently attempted to trace in Thus, the legal system needs to be changed to become detail the way a body of legal doctrine developed in a more effective instrument for delivering justice to contemporary India, I was surprised to see how the poor and disadvantaged and to do this it is neces- equivocal it was and how that equivocation was so sary to consider ways of making justice cheap, ex- solidly institutionalized. Doctrines of authority and peditious, and accessible. But again, this is justice of precedent and the hierarchic organization of courts led me to expect a body of fixed hierarchicallythe current variety. The Report continues with an eloquent char- established doctrine. But the pressures of case load, acterization of the "file-fed ... dehumanized" and turnover of judges, the system of sitting in small procrastinating character of the existing system. It benches, the limitations of legal research, and other comes out for a system of "law and justice at the factors combined to transform the law into something panchayat level with a conciliatory methodology." An very different. What purported to be a pyramidal idealized account of pre-British panchayat justice and hierarchy establishing fixed doctrine turned out to be lengthy citation of Gandhi are juxtaposed with Lenin's a loose collegium presiding over an open-textured commendation of popular participation and a former body of learning within which conflicting tendencies law minister's equation of panchayat justice with could be accommodated and elaborated. s people's courts. This suggests that the pattern of continuities and Nyaya panchayats are endorsed on the ground that discontinuities is more complex than we might infer they would remove many of the defects of the British from the overt and dramatic displacement of presystem of administration of justice since they would British law. What was displaced was, after all, not a be manned by people with knowledge of local cus- seamless monolith but a complex comprising a distoms and habits, attitudes and values, familiar with tinctive body of elevated legal learning, linked indithe ways of living and thought of the parties before 4 The indigenous content of family law, the last redoubt of them. "[T]he poor would feel that the authority which traditional law in the modern system, has been eroded by a series is administering justice to them is their own and not of post-Independence reforms. A brief account may be found in part of an alien system which they neither understand Galanter (1978). J .D.M. Derrett (1978), the leading student of nor trust." Yet the proposed panchayats do not depart these matters, contends that the traditional mold of Hindu marriage law was shattered by 1976 legislation. from current notions of law. The panchayat is to be S The example is elaborated in my forthcoming book, "Compet3
For an illuminating critique, see Baxi (1975).
DECEMBER
1978
ing Equalities: The Indian Experience with Compensatory Discrimination. "
45
rectly and unevenly to governmental practice and to the self-regulatory activities of villages, guilds, castes, and other groupings. Hindu legal learning contemplated broad delegation of regulatory functions to these groups, an expectation that was if anything exceeded in practice. s "Law" in the indigenous style included not only regulation that was local and comprehensible, but elements that were more external and opaque to the locals. The imposition of British law introduced both new legal learning and new techniques for impressing this learning on the various lesser regulatory systems. Much of the subsequent history of law in India may be seen as the interplay between the nationalization of legal activity and the continuing drive for self-regulation in many sectors of social life: local interests and understandings found new channels of expression and the new national institutions learned to accommodate diverse sorts of local regulatory activity. The tension between authoritative higher law and local)aw-ways was not introduced with British law, but was a constituent part of the earlier indigenous system. 7 Perhaps modes of combining local and "external" elements have survived the drastic changes of content. Perhaps new and distinctively Indian ways of accommodating these tensions have arisen. Or perhaps the Indian experience exhibits universal features of the accommodation of diverse normative orderings in the legal systems of complex heterogeneous societies. Certainly neither the presence of a multiplicity of normative orders nor the gap between local law-ways and the most authoritative legal doctrine are phenomena unique to India. But the sociology of law has had great difficulty in addressing these phenomena-in part because it has shadowed a tradition of normative learning which insistently obscured the visibility and centrality of these features by branding them as marginal, transient, or pathological (Galanter 1974). Hindu law, on the other hand, openly embraced normative diversity; legal learning was attuned to a multiplicity of legitimate group norms. It may well be that the older Indian learning has something to tell us about the patterns that prevail in India today. For example, indigenous notions of collegial authority or of deference to local decision makers may help us to understand patterns of regulation in contemporary India. And by extension they may help 6 The actual situation was rendered complicated beyond elaboration here by the presence of Islamic as well as Hindu legal traditions. 7 Kidder (1978) attempts to separate the foreignness and the "external" character of British law in India.
46
us to develop concepts for exploring the way in which the official, national legal systems in modern societies accommodate the variety of normative orderings which, in spite of the law's pretensions to hierarchic control, coexist with it. The relation of official law to the other normative orders may, I surmise, provide the key to a deeper understanding of law in modern societies (Galanter 1971). By learning from India's very different way of visualizing them, we can hope to illuminate these relations.
References Baxi. U pendra. "Legal Assistance to the Poor: A Critique of the Expert Committee Report." Economic and Political Weekly, 10(27): 1005-13, 1975. Baxi, Upendra and Marc Galanter. "PanchayatJustice: An Indian Experiment in Access." In Mauro Cappelletti and Bryant Garth, editors. Access to Justice: Emerging Perspectives and Issues. Leiden and Boston/Milan: Sijthoff/Giuffre (in press). Derrett. J .D.M. The Death of a Marriage Law: Epitaphfor the Rishis. New Delhi: Vikas, 1978. Gadbois, George H. Jr. "The Emergency: Ms. Gandhi. the Judiciary and the Legal Culture." Paper presented to the 16th Annual Meeting of the Southeast Conference of the Association for Asian Studies, 1977. Galanter. Marc. "The Uses of Law in Indian Studies." In Languages and Areas: Studies Presented to George V. Bobrinskoy.
Chicago: University of Chicago Press. 1967. pages 37-44. - - . "The Displacement of Traditional Law in Modern India." Journal of Social Issues, 24:65-91. 1968. - -. "Hinduism, Secularism, and the Indian Judiciary." Philosophy East and West, 21:467-87, 1971. - - . "The Aborted Restoration of 'Indigenous' Law in India." Comparative Studies in Society and History, 14:53-70, 1972. - - . "The Future of Law and Social Sciences Research ." North Carolina Law Review, 52: 1060-68. 1974. - - . "Remarks on Family Law and Social Change in India." In David C. Buxbaum, editor. Chinese Family Law in Historical and Comparative Perspective. Seattle: University of Washington Press, 1978. pages 492-97. Government of India. Ministry of Law, Justice and Company Affairs, Expert Committee on Legal Aid. ProcessualJustice to the People, 1973. [Krishna Iyer Report]. Government of India, Ministry of Law, Justice and Company Affairs. Report on NationalJuridicare: EquaIJustice-SociaIJustice, 1978. [Bhagwati Report]. Kidder. Robert. "Western Law in India: External Law and Local Response." In Harry M. Johnson, editor. Social System and Legal Process. San Francisco: Jossey-Bass. 1978, pages 155-80. Mayhew, Leon. "Stability and Change in Legal Systems." In Bernard Barber and Alex Inkeles, editors. Stability and Social Change. Boston: Little, Brown. 1971, pages 187-210. Rocher. Ludo. "Schools of Hindu Law." In J. Ensink and P. Gaeffke, editors. India Maior: Congratulatory Volume Presented to J. Gonda. Leiden: Brill, 1972. Toharia, Jose. "Judicial Independence in an Authoritarian Regime: The Case of Contemporary Spain." Law & Society Review, 9:475-96, 1975. VOLUME
32,
NUMBERS
3/4
Government Audits of Social Experiments by Ronald P. Abeles*
IN recent years both social scientists and officials of the United States General Accounting Office (GAO) have expressed concern about the problems involved in reviewing the data and the methods of social science field research. As part of its general charge from the Congress to review the expenditure of appropriated funds, the GAO has the responsibility for auditing and evaluating large-scale, policy-related programs of federally-funded social research. The scope of GAO reviews may go well beyond that of the fiscal and management audits traditionally undertaken by governmental audit agencies. The reviews are carried out not only to guard against possible fiscal improprieties; they are also intended to assess the methodological adequacy of the research. The GAO, accordingly, may evaluate the appropriateness of the research design, the manner in which the design is implemented, the adequacy of data processing procedures, and the appropriateness of the data analysis. The GAO audits are thus potentially very broad. In assuming this responsibility, the GAO has been aware of the difficulties inherent in any evaluation effort and has recently published a review of the issues involved entitled Federal Program Evaluation: Status and Issues (USGAO 1978). GAO audits of research may include reinterviews with respondents. This potential for reinterviews is a complex issue for those who are concerned with the effects of government audits upon social research. Two major problems with this procedure have been identified. First, with experimental designs generally, uncontrolled outside events that intrude upon an experiment may endanger its validity. Governmental audits, especially when they entail direct contact with participants in a study, may cause respondents either to drop out of the experiment or to answer questions with less candor. Second, government audits may violate the researcher's prior pledges of confidentiality to participants. This concern seems especially warranted when the experimental program is concerned with sensitive issues such as welfare payments or drug use. The potential invasion of the privacy of respondents raises the issue of whether respondents should be informed in advance that they might be interviewed again by the GAO. Thus, reinterviews may affect the confidentiality of information and the validity of the results obtained from field research. Accordingly, both the GAO and the Council recognized the need to examine the impact of such audits and to DECEMBER
1978
develop methods for mitigating or avoiding their intrusive effects. Under a contract from the GAO, the Council's Committee on Evaluation Research! carried out a survey of what is known about the effects of audits on the conduct of field research in the social sciences, as well as on the quality of the data obtained. It also identified and evaluated alternative methods by which the GAO might meet its legislated responsibilities in ways less potentially disruptive to canons of ethical conduct, to experimental designs, and to research results. The committee saw its review as an opportunity to offer suggestions to the research community about practices that might make the evaluation of research projects easier and more useful. The present article summarizes the committee's report and its major recommendations to the GAO.2
The impact of reinterviews Given the GAO's special interest in the effect of reinterviews (i.e., the subsequent interviewing of participants in social experiments by GAO auditors), staff members of the GAO and the Council attempted to identify as many audits containing reinterviews as possible. After reviewing recent GAO audits and con-
* Ronald P. Abeles served as Council staff for the Committee on Evaluation Research during the preparation of the report upon which this article is based. He is currently a research scientist at the American Institutes for Research (Palo Alto, California). 1 The chairman of the Committee on Evaluation Research is Robert F. Boruch, professor of psychology at Northwestern University. He served as staff for the Council's former Committee on Experimentation as a Method for Planning and Evaluating Social Intervention (1971-75), and is a coauthor (with Henry W. Riecken and others) of that committee's major product, Social Experimentation (1974), and a coeditor (with Henry W. Riecken) of a supplementary volume, Experimental Testing of Public Policy (1975) . In addition to Mr. Boruch, the members of the Committee on Evaluation Research are Howard E. Freeman, University of California, Los Angeles; Edward M. Gramlich, University of Michigan; Richard Light, Harvard University; Peter H. Rossi, University of Massachusetts; Joseph Steinberg, Survey Design, Inc. (Silver Spring, Maryland); and Harold W. Watts, Columbia University. 2 Copies of the committee's report may be obtained without charge from the GAO by requesting Audits and Social Experiments, PAD-79-I, October 1978. Requests should be directed to: U.S. General Accounting Office, Distribution Section, Room 1518,441 "Gn Street, N.W. , Washington, D.C. 20548. 47
tacting state audit agencies, the staffs found only one instance in which participants in a social research project had been interviewed first by a research team and then by the GAO. This single example, the Experimental Housing Allowance Program (EHAP), does not lend itself well to analysis or interpretation of the potential impact of reinterviews because the reinterviews took place after the researchers had completed their data collection, when they could no longer have had an impact upon the experimental results. Accordingly, it became necessary to seek indirect evidence. Some indirect evidence of the potential impact of audits was available from studies of the influence of confidentiality pledges on respondents' willingness to cooperate in research. 3 In these case studies, experiments, and surveys, willingness is typically measured (obversely) either by refusals to respond (i.e., refusals to answer any questions at all or refusals to answer particular questions) or refusals to respond truthfully. The evidence is only indirectly relevant to GAO audits, since none of the cases, surveys, or experiments involved GAO reinterviews. This evidence is pertinent in the sense that it relates to the issue of the willingness of participants to respond. This issue is particularly critical in situations in which there is a conflict between the goal of keeping research records confidential and an administrative interest in using the records. "Administrative interest" implies that an individual's record, which contains both identification information and substantive information collected by the researcher, may be used to make administrative decisions about the individual. For example, data collected by researchers might be appropriated and used by an investigatory agency to decide whether welfare payments should or should not be made to particular individuals. In this case, the appropriation of individually-identified research records for nonresearch purposes both runs counter to the researcher's assurances to the respondents and may have harmful consequences for the experiment. The committee noted that there has been an insufficient number of audits by the GAO-or by other government agencies-involving the reinterviewing of participants to reach a firm judgment on the impact of reinterviews. The available indirect evidence suggests that there is a clear risk of disrupting research, but the evidence is ambiguous as to the magnitude of the risk and the consequences of a disruption. 3 A review of this literature may be found in Boruch and Cecil (1979, in press). See especially Singer (1978).
48
Alternatives to reinterviews For the auditor with an interest in establishing the quality of interview data, reinterviews with a sample of respondents may appear to be a natural course of action. However, for a number of reasons the committee believes that the benefit of such reinterviews will be mattginal at best. First, a well-designed project will include a parallel study of the validity and reliability of its measures. For example, the veracity of responses to factual questions may be compared to already existing archival data. In such cases, reinterviews as validity or reliability checks add little, if anything, to comparisons made with existing data. Second, any major study will generate information whose quality can be assayed-up to a point-without reinterviews. The researcher's procedure for editing records, for making internal checks on the consistency of information provided by respondents, and for making comparisons of the statistical data obtained with similar data from prior studies can be reviewed by auditors without reinterviews. Third, reinterviews by outside auditors can yield ambiguous results. Ord~nary measurement error, for example, will typically produce differences between data yielded by the interview and by the reinterview. Differences in these results may also reflect actual changes in the views of the respondents that occur during the time between the two interviews. Respondents, for example, will have had time to think about a topic or issue, even a factual matter such as income, and may change their judgment about it or their interpretation of the question by the time the reinterview is conducted. Discrepancies may also appear as the result of differences in the interview procedures and the skills employed by the researchers a~d .auditors. Finally, reinterviews may produce different results merely because they are carried out under different auspices than the original interviews. Fourth, reinterviews by outside auditors raise a host of problems associated with the researcherrespondent relationship. More of the time of the respondent is required. Respondents must be told that they may be reinterviewed, which may affect participation rates. Any loss of goodwill among respondents that stems from reinterviews may lead to a public controversy and this may make others less willing to participate in similar studies in the future. Since reinterviews by auditors may produce ambiguous or irrelevant data, and since audits may disrupt social experiments, it is clear that alternatives need to be considered. The committee offered four such alternatives for the GAO's consideration. VOLUME
32,
NUMBERS
:3/4
Review of research procedures. Obviously, poor research data will be generated by poor research practices. It is also clear that for the large majority of projects an examination of the research procedures is a sufficient basis for determining whether data will be poor. Further, a critical examination of the research methods and procedures generally involves no serious privacy problems and can be undertaken without serious risk of disrupting research in the field. For these reasons the committee recommended that the GAO take every opportunity to examine research procedures before reinterviews are considered. Parallel sampling by auditors. When a main objective of the audit agency is to establish the accuracy of the sampling and of the responses, a reasonable strategy is to obtain an independent sample of the same target population used in the original survey. This strategy provides a legitimate statistical basis for judging the quality of the initial survey. In particular, it will help to determine the extent to which a sample, described as having been selected by specified methods by the researcher, is similar in characteristics to a sample selected by the GAO from the same population and using the same selection methods. Surrogate auditors. Direct reinterviews of respondents in an ongoing social experiment can disrupt the research if the reinterviews are conducted by people who are viewed with suspicion by the respondents. In particular, respondents who are able to distinguish between auditors and researchers may be less cooperative with the former, perhaps on invasionof-privacy grounds. It may be possible to cope with this problem by using a surrogate interviewing agency to reinterview a subsample of the original respondents. The results of the surrogate's survey would be provided by aggregate statistical measures, so that no identification of individual responses would be possible, "including screening the statistical results to prevent the deductive disclosure of identified responses. Subsample reinterviews by auditors and sample augmentation. Direct reinterviews may on occasion be essential to accomplish audit agency goals. For example, the GAO could choose to verify that interviews had indeed been conducted by a data collection agency and that certain responses were given in order to check the integrity of the original interviewers. Parallel sampling is normally insufficient for accomplishing these goals, and the use of a surrogate audit agency may be unacceptable to the GAO. An obvious approach to minimizing disruption of the ongoing research is to minimize the number of individuals who must be reinterviewed. The GAO, for instance, might select a probability sample from DECEMBER
1978
the existing experimental sample. Members of the subsample would then be reinterviewed by the GAO to elicit the information of interest to them. The subsample involved in the auditor's reinterviews may subsequently be of little or no use to the researchers. That is, reinterviews may provoke individuals to drop out of the study or to respond differently in the future. If these conditions prevail, the research agency can remove the GAO subsample from its sample and analyze the remaining probability sample with results properly weighted to reflect the reduced sample size. This has implications for research planning. If audit reinterviews are anticipated, and if this may have negative effects, then the sample size must be increased beyond that normally required by the research design in order to accommodate subsequent attrition as a result of the reinterviews. One major risk of this subsampling strategy is that the reinterview process may have an adverse effect on the participation rates of members of the original sample who are not reinterviewed. If the reinterviewing process takes on the character of an administrative investigation, and if members of the sample learn of the reinterviews of the subsample through the press or through informal networks, their participation rate is likely to decline.
Responsibilities of sponsors and researchers Whether the GAO will need to reinterview respondents, and what roles the GAO could most effectively take in overseeing social experiments, depends partly on the activities and responsibilities of sponsors and researchers. Consequently, the committee considered several ways in which the sponsors of social experiments and the researchers who evaluate them may contribute to an experiment's success or failure as well as to the utility of an evaluation of the experiment. Developing and implementing "Requests for Proposals." Almost without exception, social experiments and their evaluations are now undertaken in response to "Requests for Proposals" (RFPs) issued by sponsoring agencies within the government. There are three aspects of RFPs that should be considered in regard to their impact on the substantive and methodological quality of social research. First, many RFPs inadequately describe the research problem. In some cases, the description is so inadequate that the prospective researchers must create their own list of topics to be researched. Further, many RFPs fail to describe fully the policy considerations that led to the need for the social ex49
periment or program evaluation. And they often fail to specify the policy alternatives that are practical and politically feasible in the event of different types of findings resulting from the study. Without such information, it is impossible to develop appropriate study designs. Second, the time available for the preparation of good proposals based upon the RFPs can be insufficient. That is, the length of time between the advertising of the RFPs and the submission deadline for the proposals is sometimes so short that it is unlikely that any organization could develop a high quality study design. Finally, the quality of the award process can be very uneven. The judgment processes range from peerreview procedures (similar to those at NIH) to inhouse reviews, often by administrators who do not have sufficient training to make competent decisions. In addition, the awards are often made hurriedly, e.g., a sizeable proportion of RFPs end up being processed in the last months of the federal fiscal year. Monitoring projects. The project monitor is generally the only person who has the opportunity to make sure the work is done in reasonable conformity to the design, to modify the design and funding because of unexpected contingencies, and to assure the proper conduct of the project in terms of the protection of human subjects. Ideally, monitors should be persons with considerable research experience and management ability. However, monitors vary greatly in research experience, in interest in their job, and in availability to the project. This makes the use of advisory boards and consultants crucial, and both should be regarded as sharing responsibility for monitoring projects. Further, project monitors are often hindered by federal and state regulations that prevent them from being flexible, even when they and the researchers see the advantages of modifications in design, budget, or time schedule. Because of these problems, monitoring often becomes a pro forma activity, and social control opportunities as well as consultative inputs are foregone. Dissemination. The sponsors of research need to provide maximum opportunities for the dissemination of the results of their studies. At the present time, for example, final reports from projects are often difficult to obtain. The dissemination of information, including reports, is important for two reasons. First, if studies are
50
worth doing and worth supporting, then their findings should be made known to the widest possible audience. This is true whether or not the projects are "successful." Second, dissemination serves as another control mechanism for improving the quality of research. If researchers and sponsors realize that their work will be subject to public and peer scrutiny and that this scrutiny may influence their personal careers, their performance is likely to improve. Quality control of data. Another important responsibility of both sponsors and researchers is to insure that high quality data are collected and processed. However, there is no set of standards for quality that applies equally well to data collection activities for all social experiments and all social program evaluations. Although any set of standards would have to be tailored to particular settings, the committee did identify general guidelines for reasonable practice which can be used to help assure the quality of the resultant data. The committee outlined these for the GAO primarily because the researcher's quality control procedures are an important factor to be weighed by an auditor in deciding upon the scope of an audit. In particular, the better the quality control procedures used by the researcher, the less the need for auditors to reinterv!ew respondents. 4 The activities and report of the committee provide an example of how representatives of the scientific and governmental communities can cooperatively review problems raised by the use of social science research by govet:nment. It is readily apparent that governmental agencies depend heavily upon social science research for evaluations of their programs and for the development of new programs. It is also apparent that the norms, values, and goals of science are often at variance with those of government. These differences cannot be ignored, and only through continuous interchanges and inquiries such as that reported in this article can high quality social science research meet the needs of government and society.
4 Basic considerations in assaying the quality of research data are described in Riecken et al. (1974), Nunnally & Durham (1975), and many contemporary texts. Various bibliographies and proceedings of specialized meetings on quality control of data are also available (National Center for Health Services Research 1977a, b; U.S. Bureau of the Census 1974; Dalenius & Lyberg 1966; Kulley 1974; Ferber 1966; Sinaiko & Broedling 1976; Sudman & Bradburn 1974).
VOLUME
32,
NUMBERS
3/4
References Education, and Welfare, Public Health Service, Health ReBoruch, R.F., and J.S. Cecil. Assuring the Confidentiality of Social sources Administration, 1977b(DHEW Document No. (HRA) Research Data. Philadelphia: University of Pennsylvania Press, 78-3204). 1979 (in press). _ , and Henry W. Riecken. Experimental Testing of Public Pol- Nunnally,J.C., and R.L. Durham. "Validity, Reliability and Special Problems in Measurement in Evaluation Research." In E.L. icy: The Proceedings of the 1974 Social Science Research Council Struening and M. Guttentag, editors. Handbook of Evaluation Conference on Social Experiments. Boulder, Colorado: Westview Research, Vol. 1, Beverly Hills, California: Sage Publications, Press, 1975. 1975, pp. 289-354. Dalenius, T. and L. Lyberg. Bibliography on Nonsampling Problems. Institute of Statistics, University of Stockholm, Sweden, 1975. Riecken, H.W., et al. Social Experimentation: A Methodfor Planning and Evaluating Social Programs. New York: Academic Press, Ferber, R. The Reliability of Consumer Reports of Financial Assets and Debts. Urbana: University of Illinois, 1966. 1974. Kulley, A.M. The Reliability and Validity of Survey Measurement of Sinaiko, H.W. and L.A. Broedling, editors. Perspectives on Attitude Assessment: Surveys and Their Alternatives. Champaign, Illinois: Health Related Variables: A Research Bibliography. Publication 8, Pendleton, 1976. Health Services Research and Training Program, Department of Sociology, Purdue University, West Lafayette, Indiana, Singer, E. "Informed Consent: Consequences for Response Rate and Response Quality in Social Surveys." American Sociological 1974. National Center for Health Services Research. Advances in Health Review, 43: 144-162, April 1978. Survey Research Methods. Research Proceedings Series. Washing- Sudman, S., and N .M. Bradburn. Response Effects in Surveys: A Review and Synthesis. Chicago: Aldine, 1974. ton, D.C.: U.S. Department of Health, Education, and Welfare, Public Health Service, Human Resources Administration, U.S. Bureau of the Census. Indexes to Survey Methodology Literature. Technical Paper #34. Washington, D.C.: U.S. Department of 1977a (DHEW Publication No. (HRA) 77-3154). National Center for Health Services Research. Experiments in Commerce, Social and Economic Statistics Division, 1974. Interviewing Techniques: Field Experiments in Health Reporting! U.S. General Accounting Office, Federal Program Evaluation: 1971-1977. Washington, D.C.: U.S. Department of Health, Status and Issues. PAD-78-83, October 1978.
Personnel New directors and officers
Staff appointments
The Council's board of directors, at its meeting on June 17, 1978, elected five directors to serve three-year terms. Philip Jackson, psychology, University of Chicago, was reelected a director-at-large. Newly-elected directors-at-large are Jane B. Lancaster, anthropology, University of Oklahoma, and Murray L. Schwartz, law, University of California, Los Angeles. Also elected were two new directors from professional associations: Rosedith Sitgreaves Bowker, Stanford University (American Statistical Association), and Robert A. LeVine, Harvard University (American Anthropological Association). The board also elected the Council's officers for 1978-79: Otto N. Larsen, sociology, University of Washington, chairman; Samuel C. Patterson, political science, University of Iowa, vice chairman; Mr. Levine, secretary; and Mrs. Bowker, treasurer.
George Reid Andrews joined the Council staff on September 15, 1978. as staff associate for the Joint Committee on Latin American Studies. Mr. Andrews received a B.A. in history from Dartmouth College in 1972, an M.A. in history from the University of Wisconsin in 1974, and a Ph.D. in history from Wisconsin in 1978. He comes to the Council from the Pan American Society of New England (Boston), where he held the position of director of the Society'S Shattuck Library from 1977 to 1978. His major interests are in the fields of comparative and social history. Lonnie R. Sherrod joined the Council as a staff associate on September 6, 1978; he will staff the Committee on LifeCourse Perspectives on Middle and Old Age, the Committee on Work and Personality in the Middle Years, the Committee on Biosocial Science (biological foundations of parenting and offspring development), and the Committee on
DECEMBER
1978
Biological Bases of Social Behavior. Mr. Sherrod received a B.A. in psychology and zoology from Duke University in 1972, an M.S. in biology from the University of Rochester in 1974, and a Ph.D. in psychology from Yale University in 1978. At Yale, he worked with William Kessen in the Studies of Infancy program and with Jerome L. Singer at the Family Television Research and Consultation Center. Anne F. Thurston joined the Council on August 7, 1978 as staff associate for the Joint Committee on Contemporary China. Miss Thurston received a B.A. in government from Tufts University in 1965, an M.A. in political science from the University of California at Berkeley in 1966, and a Ph.D. in political science from Berkeley in 1975. She comes to the Council on leave from Fordham University, where she has been assistant professor of political science since 1974. Her special interest is in rural local politics and economic development in China.
51
52
VOLUME
32,
NUMBERS
3/4
Recent Council Publications a -pan in the Muromachi Age, edited by John Whitney Hall and Toyoda Takeshi, with the assistance of Kanai Madoka and Richard Staubitz. Papers from a conference sponsored by the Joint Committee on Japanese Studies. Berkeley and Los Angeles: University of California Press, 1977. xv + 376 pages. /
. . the May Mod ern Ch'lOese Literature 10 Fourth Era, edited by Merle Goldman. Papers from a conference sponsored by the Joint Committee on Contemporary China, August 1974. Harvard University Press, 1977.464 pages + xiii. $15.00.
Political Development in Eastern Europe, edited by Jan F. Triska and Paul M. Cocks. Product of a conference sponsored by the Joint Committee on Eastern Europe of the American Council of 'Learned Societies and the Social Science Research Council, held at Stanford University, December 4-5, 1975, New York: Praeger Publishers, 1977.374 pp. + xxiv. Hardbound, $30.00; paperback, $7.95.
.J Rural
./Modes of Perceiving and Processing Information, edited by Herbert L. Pick, Jr. and Elliot Saltzman. Papers based upon two workshops held during the spring of 1974 and 1975 and sponsored by the Committee on Cognitive Research. Hillsdale, New Jersey: Lawrence Erlbaum Associates, 1978. Distribllted by the Halsted Press Division of John Wiley and Sons.
L:king to Children: Language Input and Acquisition, edited by Catherine E. Snow and Charles A. Ferguson. Papers from a conference sponsored by the Committee on Sociolinguistics, September 6-8, 1974 at the American Academy of Arts and Sciences, Boston, Massachusetts. Cambridge, London, New York, and Melbourne: Cambridge University Press, 1977. 369 pages, including an annotated bibliography.
Small-scale Industry in the People's Republic of China, report of the American rural small-scale industry delegatio!). chaired by Dwight H. Perkins, Harvard University, whose travel was a part of the exchange program of the Committee on Scholarly Communication with the People's Republic of China and the Chinese Scientific and Technical Association. The Committee is jointly sponsored by the Council, the American Council of Learned Societies, and the National Academy of Sciences. Berkeley: University of California Press, 1977. 296 pages.
Toward a Metric of Science: The Advent of Science Indicators, edited by Yehuda Elkana, Joshua Lederberg, Robert K. Merton, Arnold Thackray, and Harriet Zuckerman. Product of a conference sponsored by the Social Science Research Council and the Center for Advanced Study in the Behavioral Sciences. New York, Chichester, Brisbane, and Toronto: Wiley-Interscience, 1978. xiv + 354 pages. $19.95.
SOCIAL SCIENCE RESEARCH COUNCIL 605
THIRD AVENUE, NEW YORK, N.Y.
10016
Incorporated in the State of Illinois, December 27, 1924, for the purpose of advancing research in the social sciences Directors, 1978-79:
IRMA ADELMAN, ROSEDITH SITGREAVES BOWKER, ROBERT EISNER,jACOBj. FELDMAN, CLIFFORD GEERTZ, PETER R. GOULD, PHILIP W.
JACKSON, FRANKLIN W. KNIGHT, GERALD H. KRAMER, JANE B. LANCASTER, OTTO N. LARSEN, ROBERT A. LEVINE, CORA BAGLEY MARRETT, PAUL H. MUSSEN, SAMUEL C. PATTERSON, MURRAY L. SCHWARTZ, ELEANOR BERNERT SHELDON, STEPHAN A. THERNSTROM
Officers and Staff ELEANOR BERNERT SHELDON, President; DAVID L. SILLS, Executive Associate; GEORGE REID ANDREWS, RONALD AQUA, ROBERT A. GATES, MARTHA A. GEPHART, ROBERTA BALSTAD MILLER, ROWLAND L. MITCHELL,jR., ROBERT PARKE,JAMES L. PETERSON, PETER B. READ, LONNIE R. SHERROD, DAVID L. SZANTON, ANNE F. THURSTON; MARTHA W. FORMAN, Assistant Treasurer; NANCY L. CARMICHAEL, Librarian
SOCIAL SCIENCE RESEARCH COUNCIL 605 Third Avenue, New York, N.Y. 10016
ADDRESS CORRECTION REQUESTED