Introduction
This Field-Tested Learning Assessment Guide (FLAG) for Science, Math, Engineering, and Technology Faculty offers a collection of methods by which to assess both student learning and student perceptions of their learning processes and other classroom experiences. It was begun as a way to bring together in accessible form the work of faculty and evaluators who are engaged in the process of reforming undergraduate science, mathematics, engineering, and technology (SME&T) education. The collection focuses on assessment methods designed by contributors from the approximately twenty NSF coalitions for the reform of undergraduate education in mathematics, chemistry, and engineering. We also include assessment methods contributed by individuals and groups beyond these consortia, and encourage site readers to submit their own work to the guide editors. We welcome comments on the efficacy of particular methods included in the FLAG that readers have tried in their own classrooms. We can thus increase the utility of the collection.
All of the methods offered by coalition members and evaluators were developed to discover how well students meet the learning objectives of courses using revised curriculum and pedagogy. There is no reason, however, why they should not be used to explore student learning, and classroom experiences, in more traditionally-taught classes. In fact, by comparing student achievement and experiences between traditional and reform courses, the SME&T education community may obtain valuable perspectives of the efficacy of our various efforts.
This is a pilot site that contains materials primarily from the chemistry community. It constitutes the seed of a significantly revised and expanded FLAG. Development of the new FLAG is the primary work of the 1998-99 College Level One Institute of the National Institute for Science Education. The revised and expanded FLAG will be unfurled in June, 1999. At that time, a link to the new site will be available at this pilot site.
Elaine Seymour, NISE, and University of Colorado, Boulder (seymour@spot.colorado.edu)
Steve Kosciuk, LEAD Center, University of Wisconsin, Madison (kosciuk@engr.wisc.edu)
Acknowledgments
The development of this FLAG was made possible by funding and practical assistance from the National Institute for Science Education (NISE), the New Traditions Chemistry Initiative (both NSF-funded projects headquartered at the University of Wisconsin-Madison), and by additional funding provided by the NSF Division of Undergraduate Education through SRI International. The NISE provided financial support to Elaine Seymour to develop the collection. The New Traditions project provided a core set of assessment tools and along with the DUE (through SRI International), provided financial support that allowed Steve Kosciuk (of the UW-Madison's LEAD Center) to design of the FLAG and construction the web-site. We thank all three organizations for making this endeavor possible.
The guide is designed both as a web-site and a hard-copy handbook. The hard-copy version reflects as much as possible the format of the website version.
How to use this guide:
Currently, there are three main ways by which to access and understand the materials in this guide.
A. As complete field-tested assessment instruments and methods of assessment
The reader is presented with a variety of annotated field-tested
instruments in their entirety as they were used by the original
developers. Included are: interview and observation protocols,
attitude surveys, and pre- and post- concept tests. These instruments
were designed to assess a variety of factors pertinent to student
learning, e.g., mastery of concepts, confidence in problem solving
abilities, integration of lecture/lab material. As such, the instruments
may be more comprehensive than specific applications would demand.
In addition, there may be sections of the instruments designed
for specific courses that would not apply outside their original
context. The reader is invited to use whatever portions of the
instruments best meet their needs. The link provided in approach
B. below is designed to assist the user in accessing those sections
of the instruments which pertain to the specific assessment needs
of the user.
B. As assessment instruments that relate to specific aspects of student learning or specific features of a given course.
The reader can select those parts of whole instruments which explore specific aspects of student learning (such as, mastery of key concepts, understanding the theory behind lab experiments), or student responses to aspects of class pedagogy (such as, group learning workshops, or labs).
Both approach A and B provide notes about each instrument or method
that include: the purpose(s) for which it was designed; the contexts
in which it has been used; evaluations of the tool's efficacy
(these materials offered by users and adapters); and contact information
for authors and other field-testers. It is hoped that this information
will allow new users to adapt assessment instruments to their
own needs, based on an understanding of the utility and value
of each assessment method and its component parts.
C. A shortcut index of all currently posted assessment tools
as well as tools whose pages are currently under development.