Joan Gillespie

IES, Chicago

Larry A. Braskamp

Loyola University Chicago

David C. Braskamp

Northwestern University Center for Urban School Policy

The process of accrediting institutions of higher education in the

U.S. embraces two principles: the system provides for self-regulation con-ducted by peers; and the starting point of the procedure is the institution’s academic mission and goals. In practice, the principle of peer assessment means that an institution “must demonstrate that it meets the collective standards of its peers” (Braskamp, Poston, Wergin, 1). With the framework of the process being the institution itself, the operative word is “standards” rather than standardization. The principle of institutional autonomy thus protects the individuality of the institution and allows for a wide variety of colleges and universities.² The evaluation of a study abroad program might be designed according to these same principles, but international and cross-cultural program components introduce three specific challenges. First, we must articulate the program’s academic mission and goals in terms of cross-cultural learning experiences, among them, language facility, field study and guided field trips, and access to local educational, cultural and political institutions. Second, we must consider non-academic program elements such as predeparture information and on-site orientation, housing, provisions for health care, and opportunities for interactions between American students and members of the host country. Third, we must negotiate the differences in academic cultures, demonstrated by pedagogical practice, student assessment, and grading.

A uniform set of educational standards and a protocol for program evaluation do not exist for most study abroad programs. The Middle States Association of Colleges and Universities is the only one of the six regional accrediting associations in the U.S. that evaluates the study abroad programs of its members as part of the accreditation process (CHE/MSA, 1). NAFSA: Association of International Educators, offers guidelines for evaluation, but is not vested with the authority to enforce them.³ Yet, the reasons that support accreditation in higher education likewise apply to the need for rigorous and systematic evaluation of study abroad programs. Among them are: assisting the prospective student in identifying an acceptable program; assisting institutions in determining the acceptability of transfer credits; creating goals for the improvement of programs; and involving faculty and staff in institutional evaluation and planning. (Sims, 42). Study abroad providers, while not accountable to the general public as are public institutions of higher education, should be held accountable to the students they enroll. Dr. Minna Weinstein, Senior Associate Executive Director, Middle States Association, explained, “Our members expend a lot of time and energy to evaluate programs for small groups of students, but it’s our responsibility to make sure programs are good even if they only serve 30 students.” Principles and policies of evaluation would contribute to the professionalism of study abroad, assuring administrators, faculty members, and students of its academic value. The net result would support the growth of study abroad, the “rapid development of the capacity to address the international dimensions of U.S. higher education” (Gagliano, 326).

This paper describes how the authors worked with a Task Force to develop the IES Model Assessment Practice (MAP), a set of detailed criteria to use in evaluating IES study abroad programs, and how IES is beginning to implement MAP. Two sets of theories provided the framework for the project: current thinking about educational program evaluation and assessment in the U.S., including current definitions of academic quality; and organizational learning, whereby the processes of planning and decision-making involve members of the organization in gathering, sharing, and interpreting information. The 18-month project was sponsored by IES, The Institute for the International Education of Students (formerly the Institute of European Studies). With this publication, we propose that study abroad practitioners join a dialogue about program evaluation and share models of good practice for the benefit of our students.

IES Programs

IES, founded in 1950, is a consortium of American colleges and universities with 50 affiliates and 75 associate members, and runs semester, full-year, and summer programs for primarily American undergraduates in 17 locations in Europe, Asia, Australia, and Latin America. IES academic programs cover a range of disciplines, including anthropology, area studies, business, economics, environmental studies, history, literature, intensive language, music theory, political science, psychology, sociology, and women’s studies. All programs offer internships in at least one field— for example, business or education—and several programs offer a variety of internships in the arts, politics, NGOs, and the media. A supporting seminar is required of students with an internship placement. Most programs are designed as full immersion, requiring two years of language instruction at the college level for admission and offering their curricula in the language of the host country. Other programs admit students at beginning and low-intermediate levels of the target language; in these cases, intensive language is required on site, and English is the language of instruction for area studies courses. Academic partnerships enable students to register at one or more local universities and, in some cases, at music conservatories and schools specializing in studio art.

Staff members and instructors are drawn from the host country; in two cases, directors are American citizens with longstanding academic affiliations in the host country. Many directors are practicing academics and teach an IES course as well as supervise the administration of their programs. All courses are taught by nationals of the host country, many of whom are full faculty members at partner institutions. Academics/profes-sionals teach courses in the arts and business.

IES programs also offer student support services for both academic and non-academic activities. Examples of the former include academic counseling regarding registration in IES courses and courses at the local university; privileges at university libraries and other local libraries; and computer facilities. Non-academic activities that are organized and supervised by IES staff members include housing placement, guided field trips, orientation, and events organized by the Center to facilitate engagement in the local culture.

IES Model Assessment Practice(MAP): Development

A Task Force was formed to develop MAP that consisted of two representatives of the IES Board of Trustees, three faculty members from IES affiliate members, and staff members of the IES Academic Department in Chicago, one of whom is one of the authors. The Task Force was led by a consultant in the field of higher education evaluation, also one of the authors. In its initial meeting, the Task Force agreed on the principles, methodology and scope of the project.

The project was conceived for the purpose of developing a set of inclusive and specific criteria to measure program quality, to be used by IES directors and staff members for improving existing programs. In addition to setting this operational strategy in motion, the project’s secondary goal was to establish a conceptual framework of quality that would serve IES in its review of existing programs and the development of new programs. The premise of MAP was to preserve the differences among program sites, or Centers, based on each program’s purpose and curriculum, the culture of the host country, and the type of student for whom the program was designed. This premise was explained by the following principles:

. High standards of quality across IES rather than standardization are the best way to achieve academic excellence at each Center. (IES, "Assessing," 2) Information was collected from the major stakeholders in IES pro-

grams—IES directors, on-site staff members, instructors, current students, and campus coordinators at IES affiliate and associate member schools. In addition, data such as program and course evaluations by students and reports of cyclical program reviews from the IES ongoing internal quality assurance system were analyzed. The most important phase in gathering information were five site visits, each undertaken by three to four members of the Task Force, to IES Centers in Paris, La Plata, Freiburg, Vienna, and Beijing, which represent the variety of IES summer, semester, and full-year programs on three continents. The reason for these extended site visits was to draw on the considerable expertise in study abroad of Center directors, staff members, and instructors who are citizens of the host country, and to use the programs as laboratories for collecting data and testing an evaluation method for study abroad.

The strategy of data collection and analysis draws from current theory in program evaluation that establishes a relationship between systematic inquiry and organizational learning. The phase of inquiry requires that employees figure prominently in the process of asking questions and gathering information, not only to benefit the organization through their expert knowledge, skills, and ideas but also to give employees a sense of ownership in the organization. This ownership ultimately will affect employees’ commitment to implementing findings and recommendations (Preskill and Torres, 12). The concept emphasizes the process of both individual and organizational learning; employees develop the habit of exchanging ideas and information and the organization puts systems into place that make use of this exchange. The planning of the IES MAP represented the beginning of a continuous process of organizational learning with regard to the quality of academic programming. The ongoing need to collect and share information, then reflect and act on this information, was written into MAP in Part IV, Interactions between an IES Center and the

4

IES Chicago office.

Task Force members determined to focus on four areas as the means for measuring academic quality: the student learning environment; student learning, assessment and the development of intercultural competence; resources for academic and student support; and the interactions among staff and faculty at IES Chicago, IES Centers, and colleges and universities in the U.S. The first three categories parallel the definition of academic quality in American higher education used by the authors and oth-ers.⁵ These terms are defined as follows:

I. Student Learning Environment.An educational institution must provide engaging and challenging country-specific learning environments for students. The quality of the experiences available to students is dependent on the structure and effectiveness of courses, out-of-classroom experiences including field study and field trips that are organized as part of the program, engagement in cultural activities, and relationships with the faculty, staff members, and other representatives of the host culture, such as host families. In a study abroad setting, the location itself is one of the most valuable resources, and the program’s ties with partner universities, academic, business, and art groups, host families, and internship sites that support a student’s cultural interactions likewise are part of the student learning environment.

II. Student Learning: Assessment and the Development of InterculturalCompetence. This category, developed over the last decade as an essential criterion of academic quality in higher education in the U.S., recognizes the dual goals of colleges and universities to enhance students’ intellectual growth and foster and develop their social and cultural competencies that prepare them to live and work in a global society.⁶ These goals both define an institution’s mission by helping it to establish priorities among desired student outcomes and are the means by which the mission is fulfilled (Braskamp, 419). Assessment also measures the “value-added,” the effects of college upon students’ cognitive and affective development (Nordvall, 485). “Talent development” is used interchangeably with “value-added” and helps define the relationship between mission and assessment—that is, if an institution’s mission is to develop the talents of students and faculty, then forms of assessment should be integrated into the teaching and learning process and be guided by the kinds of talent that the institution fosters (Astin, 158; 171). In the development of intercultural competence, this principle opens the way for the use of qualitative measures such as interviews in the target language or an artistic project to assess a student’s level of interaction with the host culture and understanding and acceptance of cultural difference.

III. Resources for Academic and Student Support.An educational organization must have an infrastructure and financial and human capital to provide a high quality learning environment. Criteria are determined in part through quantitative measurements: the ratio of on-site staff to students, library collections, computer facilities, admission standards, the grade point average of enrolled students, and tutorial assistance. Qualitative measures include the credentials of faculty, staff, advisors and health and safety guidelines.

IV. Interactions among Staff and Faculty at IES Chicago, IES Centers, andcolleges and universities in the U.S. While criteria are specific to the administrative framework of IES, it might be enlarged to cover an institution’s level of communications with a variety of publics, such as government and educational agencies at home and abroad.

The Task Force identified several issues to investigate during the site visits in order to determine the criteria of quality assurance according to the four categories. The question of balance was critical in defining three issues in particular: academic cultures, student activities, and program resources. Through observation of classes and interviews with faculty members, the Task Force considered the differences in academic cultures, that is, the need to introduce American students to the teaching styles and assessment practice of the host country while acknowledging American students’ learning styles, particularly expectations of specific class assignments and a high level of interaction with instructors. Interview questions with instructors included the following:

What are your impressions of American students’ learning habits?

How would you characterize the level of students’ academic and cultural preparation for a semester in the host country?

Did you make changes in your syllabus based on the students’ preparation and comprehension?

What are the field study assignments and how do they immerse students in the host population and its culture?

Do you hold office hours or provide conference time for students?

Another topic concerned the range of opportunities for cultural integration that are provided to students that, at the same time, encourage their independence and resourcefulness. A third topic focused on the extent to which facilities and resources (particularly computer and educational technology) should approximate available technology on U.S. campuses. These issues were examined through interviews with staff, informal meetings with students, a tour of facilities, including facilities at partner institutions, housing, and a cultural activity with students or an activity or event similar to one that students experience during the semester.

One other source of information for compiling the criteria of Model Assessment Practice was a survey of study abroad coordinators at IES affiliate and associate schools on their perspective of academic quality. This survey supplied data on reputation, one of the traditional indicators of quality. The reliability of the data depends on those making the judgments, the criteria they use and their access to evidence that supports the criteria (Nordvall, 484). By querying those with the greatest knowledge of study abroad programs, the Task Force received data with a high degree of reliability.

MAP in its final form employs the four categories identified at the beginning of the project (see Appendix). Criteria in each sub-category are descriptive rather than prescriptive, in keeping with the principles that guided the project to protect the curricular and cultural diversity of the Centers and to promote standards, not standardization. For example, criteria listed under “Instructional Quality” state: “Class experiences make effective use of location,” and: “Language instruction makes effective use of field study and local cultural institutions.” The term “effective use” allows for latitude in a Center’s decisions regarding the structure and frequency of these experiences, depending upon the overall curriculum and the content of individual courses. Students evaluate the “effectiveness” of these experiences in program and course evaluations.

MAP consists of four other evaluative procedures, three consisting of formative data and one of summative data.⁷ The criteria for quality assurance served as the conceptual framework for course evaluations and general evaluations of a program, both completed by students. These evaluative instruments already were in use and were revised from two perspectives: to reflect the assessment criteria, thereby giving students the opportunity to indicate how well they are being met, and to offer instructors and program staff the means to assess students, albeit anonymously. Course evaluations, focusing on the instructional process and the cultural context, question students about their understanding of course content, the academic rigor, and the use of the local environment. While traditional assessment tools such as course examinations and research assignments also measure a student’s understanding of content, they do not reveal a student’s expectations of academic rigor or his or her insights into the local culture. By making these assessments, course evaluations are intended to be a planning aid to instructors to balance course content with student expectations of knowledge and comprehension, which in turn facilitate cultural analysis. This use focuses academic quality on academic processes in the classroom and places part of the responsibility for improving academic quality in the hands of instructors (Nordvall, 486).

The general program evaluation serves as a form of student self-assessment, particularly in the categories of interpersonal and intrapersonal skills. Questions address a student’s independence, level of adaptation, understanding of his/her own cultural values, self-knowledge, prob-lem-solving skills, and tolerance of diversity. These affective and behavioral areas of growth are central to the experience of study abroad, yet are difficult to quantify. These evaluations are analyzed in tandem with evaluations conducted by home institutions when their students return to campus.

Finally, the assessment criteria serve as the basis for the in-depth five-year reviews of each program, conducted by a committee composed of faculty representatives of affiliate members and of instructors and students at the Center. These program reviews have been in place at IES since 1974, and will continue as an essential step in monitoring and improving academic quality.

A set of summative data was developed as part of the ongoing system of program evaluation by program directors and the Chicago office. Statistical profiles of each Center summarize the most significant information on student admissions, revenues and expenses, the academic program, human resources, and physical resources. Purely quantitative, the profiles offer the basis for making administrative decisions about admissions, recruiting, programming, staffing, space needs, and funding. The profiles are updated each semester and their use for planning purposes applies in most cases to the coming academic year.

Implementation of MAP

After the criteria of quality assurance were drafted, the process of collecting data and analyzing information continued, in keeping with the concept of organizational learning. The next step was a self-study according to the criteria that was conducted by program directors and their staff. Because MAP allows for differences among cultures and programs, the criteria do not constitute a grading system or even a checklist. Instead, the self-study requires that program directors and faculty focus on those criteria that apply to their program, report on how they meet the criteria, and, if they do not meet them, how they would propose to do so.

The first section, Student Learning Environment, contains two major divisions, one related to academic programming and another focusing on out-of-classroom activities that are devoted to a student’s cultural integration. The criteria devoted to Instructional Quality generated responses in the self-study that describe the difficulty of negotiating between American students and faculty of the host country, each group with its expectations of student preparation and teaching methods. IES Dijon explained: “Faculty have been hesitant about setting their level of expectation. They find that students have the potential but are easily distracted by the environment” (IES, “Self-Assessment,” 4). IES Freiburg responded: “Faculty have to consider very diverse language levels of students.… Faculty-staff meetings are held once per semester, where teaching methods and the question of standardization (how Americanized can/must a course be without jeopardizing educational goals like intercultural academic competency) are discussed” (IES, “Self-Assessment,” 5). The challenge of negotiating the differences between students and faculty led to the decision at IES Chicago to discuss this issue in a meeting of all directors and reach agreement about balancing the needs and expectations of both parties.

Self-assessments in another category in Student Learning Environment, Curricular Design and Integration, revealed the extent to which courses take advantage of the students’ presence in the country to enhance their knowledge of political, cultural, and social life. Freiburg and Vienna specifically identified enrollment in partner institutions as “an absolute guarantee that students become integrated in community life as far as their individual social skills permit” (IES, “Self-Assessment,” 8), a statement that confirms the priority given to academic advising for university enrollment in full-immersion programs. This category also underscored the differences between programs and the relevance of standards, not standardization. In the self-assessments according to the criterion, “Course content and curriculum adequately reflects the variety of cultures of the host country,” IES La Plata responded: “Argentinian common sense and academic sense do not consider this as the definition of Argentinian culture” (IES, “Self-Assessment,” 9). Similarly, IES Vienna wrote: “A variety of cultures in the American sense does not really exist in Austria,” and IES London explained: “It is more important that students learn the dominant culture first, then diverse cultures. In England, the challenge is to look outside London; London itself is diverse but not representative of the rest of Britain” (IES, “Self-Assessment,” 9). Thus, while the criterion might not apply to all countries, its inclusion is significant, particularly for countries such as France and Germany that are addressing the political, social, and economic challenges of immigrant communities.

One theme in the four major categories of MAP focuses on the development of skills related to intercultural competence, both through coursework and through out-of-classroom experiences (Moore and Ortiz, 47-50). This focus is consistent with the IES mission and its current efforts to develop program support to foster these skills through cognitive, interpersonal, and intrapersonal growth. The targeted outcome is that students develop the ability to understand and critique their own culture. Thus, one of the criteria is: “Approaches to course material should incorporate an analysis of cultural differences.” Several of the self-assess-ments were positive, even if such analysis does not constitute a formal aspect of the curriculum. For example, IES Tokyo responded: “The incorporation of cultural differences into class is a very natural extension of the course material and is a frequent topic of student discussion” (IES, “Self-Assessment,” 11). Another criterion in the category of internships stipulates: “Internships help develop intercultural cognitive and interpersonal skills.” The response of IES London drew on student evaluations as the basis of its self-assessment that “this development occurs. Internships offer a concentrated view of British behaviour, and students must adapt a different perspective and analytical framework” (IES, “Self-Assessment,” 14).

Two other programming elements that address cross-cultural learning and skills in MAP are on-site orientation and re-entry. The first criterion of orientation stipulates that the Center provide information on gender and race relations in the host country. While several programs responded affirmatively, IES Nantes explained, “Students are informed about gender relations, but race has not yet been discussed with the entire student body. Race relations are dealt with during relevant IES classes or on a case by case basis” (IES, “Self-Assessment,” 2). An example of this alternative approach in Nantes is an assignment in language classes that requires students to discuss race relations in France with their homestay families, which meets the intent of the criterion. A re-entry program was identified as a priority by some Centers that do not currently offer a formal session on the challenges and opportunities presented by reverse culture shock. Further, mutual support in developing programs on orientation and re-entry are included in Part IV, Interactions among IES Chicago, IES Centers, and colleges and universities in the U.S.

Two sections of Part III, Resources for Academic and Student Support reflect the current climate of U.S. higher education and study abroad. One of these covers the topic of health and safety. Student insurance coverage, student access to health care, instructions to students for handling a medical emergency, and enforcement of the IES policy on sexual harassment are among the criteria that Centers must meet. The second sub-category is Instructional Technology, available both to students and instructors. Guidelines provide for student access to computers, e-mail, and digital resources such as the Internet, and technology for faculty use. This latter set of criteria represents a significant funding item at Centers where students do not have access to computer facilities at partner universities. Options such as shared facilities are being investigated at such programs.

One goal of the self-study is that program staff members continually address the criteria that are relevant to their purpose, curriculum, and students. A time limit for meeting goals is not imposed; as guidelines, certain criteria of MAP must be monitored constantly; meeting goals one year would not presume they are met the following year. The second goal of the self-study is that program staff members view MAP as a work-in-progress, that is, they are expected to add new criteria over the course of the years to reflect a program’s changing priorities. In keeping with these goals, a complete self-study will be required of program sites in advance of a cyclical review, a phase similar to the self-study that initiates an accreditation review at a college or university department or program.

Some Centers used the self-study as the basis for funding priorities for the coming year, 1999-2000, particularly to enhance the Student Learning Environment. Curriculum development includes such initiatives as Music History and Theory courses in Vienna, an extended Orientation with intensive language study in Salamanca to facilitate enrollment in Universidad de Salamanca, and revised language materials in Beijing, with training in their use for instructors. Activities other than coursework that support cultural integration, including interaction with students in the host country, field study, and membership in academic or professional associations for the benefit of students also count among funding priorities. IES Madrid will initiate a Cinema Forum that will enlarge the program of intercambios to include viewings and discussions of classic Spanish films with a professor of film at Universidad Complutense de Madrid. IES Paris is organizing a required field trip to Marseilles and an optional field trip to either Senegal or Benin, to complement the new Francophone Studies curriculum. In London, an invitation to join the Royal Institute of International Affairs is being funded, which will give students the opportunity to attend public lectures and use the private library.

Conclusion

The goals of this project were twofold: to develop a product—a set of criteria of quality assurance—and to put in place a process for continual systematic evaluation, both self-evaluation and the periodic program reviews by outside evaluators. As the project evolved, it also drew on the principles of organizational learning, specifically, involving employees in planning and implementing change. In theory, involving an organiza-tion’s members in the evaluation contributes to the individual’s and the organization’s learning (Torres, Preskill and Piontek, 3). The development of MAP challenged staff members to think specifically about the meaning of academic quality in study abroad programs and methods for maintaining and improving academic quality. Conversely, the criteria also offer a framework for defining academic quality in terms of the goals of the organization and the specific mission of each Center.

Organizational learning will continue to develop through the self-study conducted by Center staff members in conversation with IES Chicago staff members and through student evaluations. “As new learning emerges from collecting, analyzing and interpreting data, the evaluative inquiry process is recycled by asking additional questions, identifying values, beliefs and assumptions and reflecting on, and engaging in dialogue, about the findings. The continuous process of evaluative inquiry further nourishes individual, team and organizational learning” (Preskill and Torres, 3). In this particular setting, the process carries implications for the organization’s strategic planning for academic quality. The criteria of MAP serve as talking points for allocating financial resources; that is, the evaluation of academic quality is directly tied to decisions to provide funds to enhance academic quality. This causal connection between evaluation and the allocation of resources likewise establishes a link between resources and the quality of the student learning environment. In a climate of organizational learning, the organization’s members—staff, faculty, and students—who engage in inclusive discussions about academic quality are the ones who benefit.

Notes

¹ The authors wish to thank the members of the IES Board of Trustees for providing support for this project through the endowment fund and Mary Dwyer, President, IES, for her leadership and vision. The authors also acknowledge with gratitude the assistance of the members of the Task Force: IES Board members Stanley Katz, Woodrow Wilson School, Princeton University and Kathryn M. Moore, College of Education, Mighigan State University; affiliate representatives David Deal, Department of History, Whitman College; Pamela Ferguson, Department of Mathematics, Grinnell College; Mark Ruhl, Department of Political Science, Dickinson College; and Michael Steinberg, Vice President for Academic Programs, IES.

² Larry Braskamp, Muriel Poston, and Jon Wergin identify four systems that work together to insure public accountability and the improvement of academic quality: internal campus reviews; governmental laws, rules and regulations; the marketplace; and accreditation. Of these four, internal reviews and accreditation focus on individual institutions. “By respecting unique institutional and program missions accreditation has preserved what has become the envy of the world: a rich diverse array of colleges and universities…. Thus accreditation has nurtured intellectual capital, the pursuit of scholarship relatively unfettered by external politics, extreme pragmatism, and power” in “Accreditation: Sitting Beside or Standing Over?” (CHEA, 1997) 1-2. Serbrenia J. Sims identifies the origin of this principle as the North Central Association, which determined in the 1930s to judge an institution “in terms of its purpose and total pattern as an institution,” thus making it possible to adapt the accrediting process to a variety of schools in Student Outcomes Assessment: A Historical Review and Guide to Program Development (New York: Greenwood Press, 1992) 69-70.

³ Ronald Pirog and Patricia Martin focus on programs sponsored by

U.S. institutions of higher learning, but acknowledge that “there are so many different types of (overseas education) programs in existence, ranging from U.S.-planned and run traditional ‘island’ programs and branch campuses to direct enrollment in foreign universities and exchanges—as well as experiential, service-learning, and internship programs. This superabundance of numbers and types makes the task (of evaluation) extremely difficult—especially if one wants to compare the quality of one to another” in NAFSA’s Guide to Education Abroad for Advisers and Administrators 2nd Ed. (Washington, D.C.: NAFSA, 1997) 333.

⁴ David R. Schwandt defines organizational learning as an institutional way of life, one that makes long-term contributions to the organi-zation’s health and adaptability. Based on the idea of the organization as a social system, the concept applies to the particular system “of actions, actors, symbols, and processes that enable an organization to transform information into valued knowledge which in turn, increases its long-run adaptive capacity” in “Learning as an Organization: A Journey into Chaos.” Ed. Sarita Chawla and John Renasch. Learning Organizations: Developing Cultures for Tomorrow’s Workplace (Portland: Productivity Press, 1995) 370. Rosalie T. Torres, Hallie S. Preskill and Mary E. Piontek explain the theory in practice as “a continuous process of organizational growth and improvement that (1) is integrated with work activities; (b) involves the alignment of values, attitudes, and perceptions among organizational members; and (c) uses information or feedback about both processes and outcome to make changes” in Evaluation Strategies for Communicating and Reporting (Thousand Oaks, Cal.: Sage, 1996) 2.

⁵ Larry A. Braskamp and David C. Braskamp explain the current philosophy that “colleges and universities are now being conceptualized as learning communities—communities in which the most important persons are the students as learners. Faculty and the surrounding learning environment—the curriculum, social activities, facilities—exist to foster the social, physical, spiritual, and the intellectual development of students… When standards are connected to student performance they provide a very compelling argument for refocusing the definition of quality in higher education. By linking standards and performance, student learning and development becomes the starting point for examining program quality (i.e., program and institutional effectiveness)” in “The Pendulum Swing of Standards and Evidence” (CHEA Chronicle 5, 1997) 1. Alexander Astin similarly advances the concept of academic quality in terms of “enhancing the intellectual and scholarly development” of both faculty and students. “The most excellent institutions are, in this view, those that have the greatest impact—‘add the most value,’ as economists would say—to the students’ knowledge and personal development” in Assessment for Excellence: The Philosophy and Practice of Assessment and Evaluation in Higher Education (NY: American Council on Education, Macmillan Publishing Co., 1991) 6.

⁶ Patrick T. Ewell references a joint project of the National Center for Higher Education Management Systems and Education Commission of the States that queried corporate and political leaders in focus groups on the issue of “what the results of four years of college ought to be.” Among the characteristics identified by the groups was “the ability to bridge cultural and linguistic barriers. While in some ways resembling the academy’s own concern with cultural diversity, the emphasis here was again far more applied. At its center was the ability to work effectively with people drawn from different backgrounds and value systems, far more than simply personal tolerance for ethnic and national differences” in “Assessment of Higher Education Quality: Promise and Politics.” ed. Samuel J. Messick, Assessment in Higher Education; Issues of Access, Quality, Student Development and Public Policy (Mahwah, N.J.: Lawrence Erlbaum Associates, Publishers, 1999) 151-152.

⁷ Sims describes the difference between summative and formative data. Formative data, that gathered by the Center self-assessments and student evaluations of courses and programs, provides “information about how a program operates and how to improve it” and the main audiences are program staff and managers. The main use of summative data, the Center profiles, is administrative: that is, to aid in decisions such as the allocation of funds, for audiences that include the organization’s leaders and sponsors (86).

APPENDIX

IES MODEL ASSESSMENT PRACTICE

I. Student Learning Environment

A. Pre-departure information

B. Cultural orientation

C. Instructional quality

discuss pedagogy, course content, and student progress

D. Curricular design and integration

E. Curricular focus on intercultural competence

F. Language development opportunities

G. Internships

1. Internships give IES students the opportunity to participatein and to critically observe a segment of the work force in the host country

H. Field study and trips

I. Engagement in IES-sponsored cultural and social activities

1. The Center organizes activities that facilitate student engagement in the local culture

J. Involvement in other academic, political, economic and culturalinstitutions

K. Reentry into home culture and home institution

E. Intrapersonal growth

IV. Interactions among IES Chicago Office, IES centers, and colleges and univer-sities in the United States.

A. Quality of communications

B. Involvement of U.S. college faculty

C. Efficiency of recruitment and enrollment procedures

D. Professional development

References

Astin, Alexander W. “Assessment, Student Development, and Public Policy.” Assessment in Higher Education: Issues of Access, Quality, Student Development, and Public Policy. Ed. Samuel J. Messick. Mahwah, N.J.:

Lawrence Erlbaum Associates, Publishers. 1999. 157-175.

-----. Assessment for Excellence: The Philosophy and Practice of Assessment and Evaluation in Higher Education. New York: American Council on Education, Macmillan Publishing Co. 1991.

Braskamp, Larry A. “Purposes, Issues, and Principles of Assessment.” NCA Quarterly 66 (1991): 417-429.

----- and David C. Braskamp. “The Pendulum Swing of Standards and Evidence. CHEA Chronicle 5 (1997): 1-4.

-----, Muriel Poston, Jon Wergin. “Accreditation: ‘Sitting Beside” or ‘Standing Over’?” CHEA (1997): 1-2.

Commission on Higher Education, Middle States Association of Colleges and Schools (CHE/MSA). Manual for Study Abroad Evaluations. Philadelphia: CHE/MSA, 1994.

Crow, Steven D. “International Education and Institutional Accreditation.” NCA Quarterly 63 (1988): 379-87.

Ewell, Peter. “Assessment of Higher Education Quality: Promise and Politics.” Messick 147-156.

Gaglione, Felix. “Globalization of the University.” NCA Quarterly 67 (1992): 325-334.

House, Ernest R. Professional Evaluation: Social Impact and Political Consequences. Newbury Park, Cal: Sage, 1993.

Hudgins, James L., Dorcas A. Kitchings and Starnell K. Williams. “Laying the Foundation for Effectiveness through Assessment: Midlands Technical College.” Assessment in Practice: Putting Principles to Work on College Campuses. Eds. Trudy W. Banta, Jon P. Lund, Karen E. Black, Frances W. Oblander. San Francisco: Jossey-Bass Publishers. 1996. 325-28.

IES, Institute for the International Education of Students. “Assessing the

Academic Quality of IES Centers.” Unpublished essay. 1999.

-----. “IES Model Assessment Practice.” 1999.

-----. “IES MAP: Center Self-Assessment.” 1999.

Moore, Katherine M. and Anna M. Ortiz. “The Intercultural Competency

Project: Site Visit and Focus Group Report.” Unpublished essay. 1999

Nordvall, Robert C. and John M. Braxton. “An Alternative Definition of Quality of Undergraduate College Education: Toward Usable Knowledge for Improvement.” The Journal of Higher Education. (67):

483-497.

Pirog, Ronald, Michael Laubscher, and Patricia C. Martin. “Program Evaluation.” NAFSA’S Guide to Education Abroad for Advisers and Administrators. 2nd ed. Eds. William Hoffa and John Pearson. Washington, D.C.: NAFSA Association of International Educators. 1997. 333-349.

Preskill, Hallie and Rosalie T. Torres. Evaluative Inquiry for Learning in Organizations. Thousand Oaks, Cal.: Sage, 1999.

Schwandt, David R. “Learning as an Organization: A Journey into Chaos.”

Learning Organizations: Developing Cultures for Tomorrow’s Workplace.

Eds. Sarita Chawla and John Renesch. Portland: Productivity Press. 1995. 364-379.

Sims, Serbrenia J. Student Outcomes Assessment: A Historical Review and Guide to Program Development. New York: Greenwood Press. 1992.

Torres, Rosalie T., Hallie S. Preskill and Mary E. Piontek. Evaluation Strategies for Communicating and Reporting. Thousand Oaks, Cal.: Sage. 1996.

Weinstein, Minna. Telephone Interview. 19 February 1999.

Wolf, Richard M. Evaluation in Education: Foundations of Competency Assessment and Program Review 3rd ed. New York: Praeger. 1990.