SJSU end-of-semester survey
FLAG homepage
-
Overview of the tool/method
This tool was originally designed by researchers at the LEAD Center
at the University of Wisconsin-Madison for the New Traditions Chemistry
curriculum reform initiative. It was used with slight modifications in
both Fall and Spring semesters at San Jose State University (a California
four-year State college). The survey assesses student perceptions of the
impact of a broad variety of factors related to the processes by which
they learn chemistry including: confidence in their ability to master fundamental
aspects of chemistry; as well as their perceptions of what were the most
effective strategies by which they solved chemistry problems and grasped
chemistry concepts. Additional questions on this survey were designed to
assess outcomes related learning processes which also rely on student self-report
(e.g., students' perceptions of science as a career; ability to work in
groups; enjoyment of science). The tool is designed to be given at the
end of the semester. Although, the survey was originally designed for general
chemistry courses many of the questions are equally appropriate for higher
level courses as well.
Return to top of page
-
Analysis/interpretation of findings
Perhaps, the most basic point to realize regarding the analysis of
Likert scale type survey questions (i.e., questions of the type strongly
disagree=1,…, strongly agree=6) is that the scale (in this case from 1
to 6) is ordinal as opposed
to metric. In other words, a
response of 5 can be assumed to be higher than a response of 3, but not
necessarily 5/3(=1.667) times higher than 3. Moreover, there is absolutely
no guarantee that the contrast, say, between 5 and 3, for one respondent
is the same as that for another respondent. On the contrary, one can be
virtually certain that each respondent has their own "scale." Nevertheless,
one may observe interesting contrasts between different groups of respondents.
For example, one may find (as we did) that male freshman chemistry students
tend to report higher confidence levels (higher percentages of 5s and 6s)
than female students at the beginning of the semester, but that this gap
(not surprisingly) narrows by the end of the semester. In general, questions
framed in a scale "invariant" way will be both more reliably answered,
and the answer more easily interpreted. Thus, although "statistically significant"
differences in average responses may be a useful method to "flag" potentially
interesting contrasts the interpretation of such differences will be of
one sort in the case that a large proportion of one group responds at the
top of their scale(s) (in comparison to the other group), and of quite
another sort in the case where most of the responses of both groups are
near the middle of their scale(s). [The implications for statistical analyses
are also far reaching. In particular, it means that the more appropriate
statistical methods (i.e., the methods with p-values that are more reliably
and easily interpreted) are those which are "invariant" under arbitrary
monotone changes of scale. Thus, rank tests, permutation tests, chi-square
test, and log-linear methods are more appropriate than "normal theory"
tests such as t-tests or Analysis of Variance (ANOVA) methods.]
Return to top of page
-
Sample results
SJSU
Fall 1995 Chem 30A end-of-semester survey
Return to top of page
-
Who designed and tested the tool
Professor Maureen Scharberg at San Jose State University adapted this
survey with slight modifications from a similar survey used by the New
Traditions chemistry curriculum reform initiative at the University of
Wisconsin-Madison. Professor Scharberg can be contacted at:
-
Professor Maureen Scharberg
Department of Chemistry, San Jose State University
San Jose, CA 95192-0101
email: scharbrg@pacbell.net;phone:
408-924-4966
-
Name and location of original designer(s)and tester(s)
The survey was originally designed by Dr. Steve Kosciuk, LEAD Center
statistician, and implemented by many of UW-Madison Chemistry faculty,
who taught freshman chemistry courses from Fall 1995 through Spring 1997.
For further information contact:
-
Professor John Moore
Department of Chemistry, University of Wisconsin-Madison
1101 University Ave. Madison WI 53706
rm. 1321 Chemistry bldg.
email: jwmoore@chem.wisc.edu;
Phone: 608-262-5154
-
Dr. Steve Kosciuk,
LEAD Center, University of Wisconsin-Madison
1402 University Ave., Madison, WI 53706
email: kosciuk@engr.wisc.edu;phone:
608-265-5926;
LEAD Center administrative office: 608-265-5920
Return to top of page
-
Others who have used this tool
-
Professor Steve Branz
Department of Chemistry, San Jose State University
San Jose, CA 95192-0101
email: branz@leland.stanford.edu;phone:
408-924-4999
-
Professor Kim Kostka
Department of Chemistry, University of Wisconsin-Rock County
email: kkostka@mail.uwc.edu;phone:
608-758-6532
Return to top of page
Return to top of page
