PROGRAM EFFECTIVENESS: STUDENT LEARNING OUTCOMES
ULEAD
ASSESSMENT OF PSYC 2301
Dan
B. Dydek, Ph.D.
Associate
Professor of Psychology
María
Cisneros-Solís,Ph.D.
Professor
of Psychology
June
5, 2003
PROGRAM
EFFECTIVENESS: STUDENT LEARNING OUTCOMES
ULEAD
ASSESSMENT OF PSYC 2301 (FY2002-2003)
Purpose/Goal
The Department of Psychology at ACC
values excellence in the quality of our courses and delivery of instruction.
Continual improvement of instruction is, therefore, a major goal of this component
of our program effectiveness review process.
As a means to achieve continual
improvement, we select a different course each year to evaluate as a component
of our program effectiveness review process. This academic year (2002-2003) we
have selected PSYC 2301, Introduction to Psychology. This course is required
for an Associate’s and Bachelor’s Degree in Psychology, as well as in several
other majors. It transfers to most universities in Texas and the content
knowledge and skills covered within the course will enhance the students’
success in their chosen careers and everyday life situations.
Learning Outcomes
The learning outcomes of PSYC 2301
incorporate a comprehensive overview of psychological principles and concepts,
which students can apply to their chosen careers and everyday life
situations. The specific core topics
including representative theories and research that need to be mastered by our
students include the following:
·
Research
Methods
·
Personality
·
Learning
·
Life-Span
Development
·
Physiological
Basis of Behavior
·
Cognition
(including Sensation, Perception & Memory)
·
Stress
(and intellectual, social, physical and emotional health)
·
Psychological
Disorders
·
Social
Psychology
Prior to the beginning of the fall, 2002 semester
the Psychology faculty within the Behavioral Sciences Task Force decided on the
following Specific Learning Outcomes and Assessment Criteria for PSYC 2301,
Introduction to Psychology:
Specific
Learning Outcomes:
Upon completion of the course the majority of
students will demonstrate the acquisition of knowledge and mastery of the “Core
Topics” for the Introduction to Psychology course (PSYC 2301).
Assessment
Criteria:
In a test given at the end of the course, 70% of
students quizzed on a multiple-choice post-test will answer correctly items
specific to each of the nine “Core Topics” in PSYC 2301.
Methodology
In order to assess the extent to which the “Core
Topics” are being effectively taught and learned in the Introduction to
Psychology course, PSYC 2301, the following methodology was adopted. Ten lecture sections of PSYC 2301, out of a
total of 48 sections, were randomly selected from the spring, 2003 Course
Schedule. Of the ten instructors
teaching these sections, nine agreed to participate in this course evaluation
process by administering the PSYC 2301 Assessment Quiz (See Appendix.) at the
end of the semester. Of these nine,
eight instructors obtained the informed consent of their students and
administered the quiz during the last two weeks of the semester. Students participated anonymously.
The students answered the eighteen item multiple
choice quiz on Scantron forms which were aggregated, scored, and item analyzed
to determine the percentage correct on each of two items relevant to each of
the nine Core Topics. 174 Scantron
forms were returned to be scored and item analyzed. Note that all participating instructors remain anonymous at the
time of the analysis and report.
Results
Table 1.0 shows the number and
percentage of students who incorrectly (and correctly) answered each item on
the Assessment Quiz. Also shown is the
Mean % Correct for Each Core Topic, which was obtained by calculating the mean
of the percent correct scores on the two items representing each Core Topic. Figure 1.0 also shows the Mean
Percentage Correct for each Core Topic.
Table 1.0
|
Item # |
# missed |
# correct |
% correct |
Mean
% correct per topic |
||
|
1 |
84 |
90 |
51.7 |
Research Methods (Items 1 & 2): |
||
|
2 |
47 |
127 |
73.0 |
62.4 |
||
|
3 |
14 |
160 |
92.0 |
Personality (Items 3 &4): |
||
|
4 |
68 |
106 |
60.9 |
76.4 |
||
|
5 |
97 |
77 |
44.3 |
Learning
(Items 5 & 6): |
||
|
6 |
71 |
103 |
59.2 |
51.7 |
||
|
7 |
87 |
87 |
50.0 |
Life-Span Development (Items 7 & 8): |
||
|
8 |
64 |
110 |
63.2 |
56.6 |
||
|
9 |
54 |
120 |
69.0 |
Physiological
Basis of Behavior (Items 9 & 10): |
||
|
10 |
61 |
113 |
64.9 |
67.0 |
||
|
11 |
30 |
144 |
82.8 |
Cognition
(Items 11 & 12): |
||
|
12 |
47 |
127 |
73.0 |
77.9 |
||
|
13 |
115 |
59 |
33.9 |
Stress
& Health (Items 13 & 14): |
||
|
14 |
16 |
158 |
90.8 |
62.4 |
||
|
15 |
35 |
139 |
79.9 |
Psychological
Disorders (Items 15 & 16): |
||
|
16 |
40 |
134 |
77.0 |
78.4 |
||
|
17 |
107 |
67 |
38.5 |
Social
Psychology (Items 17 & 18): |
||
|
Figure 1.0 |
124 |
50 |
28.7 |
33.6 |
These data indicate
that the Assessment Criteria pre-established by the Task Force were attained
for three “Core Topics”: Psychological Disorders, Cognition (Including
Sensation, Perception & Memory), and Personality, with 78.4%, 77.9% and
76.4% of students answering relevant items correctly. The next three Core
Topics in terms of degree of mastery displayed on this assessment quiz were the
Physiological Basis of Behavior, Stress and Health and Research Methods, all of
which were between 60% and 70% of students answering the assessment quiz items
correctly.
The three weakest areas, as evidenced by this
assessment methodology, are Life-Span Development, Learning, and Social
Psychology, which were respectively 56.6%, 51.7%, and 33.6% of students
answering items correctly.
Interpretation of Results
These results appear to indicate that the assessment of this course needs to be revisited in the near future. A revision of the methodology utilized to measure student learning mastery is needed due to the following limitations of this evaluation study:
· Limited sample size (small number of participating lecture sections)
· Limited number of items (2 test questions per topic) representing each core topic
· Assessing student mastery at the end of a semester still in session, while attempting to minimize the intrusion into the teaching and learning process itself. For example, it might actually be the case that the low percentage of students attaining the minimum criterion performance in the area of Social Psychology has to do with the fact that most PSYC 2301 classes cover that topic during the last weeks of the course.
· Learning and mastery of psychological principles cannot be measured validly by one type of instrument, i.e. a multiple-choice test.
These results also highlight the possibility that
the faculty that were randomly selected may not have been aware of the core
topics designated by the Psychology Department and thus did not cover them
comprehensively in their classes.
There is no way to ascertain the causes
of these measured outcomes based upon these data. All one can reliably conclude is that the
criterion of 70% of students achieving knowledge and mastery on each of the nine Core Topics has not
been demonstrated with this methodology.
A reasonable inference might be that continued efforts to improve our
departmental effectiveness in PSYC 2301 would not be contraindicated. That conclusion is certainly compatible with
our departmental goal of continual improvement in all of our course offerings.
Implementation of Improvements
These results lead to the conclusion that the criterion of 70% of students achieving knowledge and mastery on each of the nine Core Topics has not been demonstrated with the methodology used in this assessment study. It, therefore, appears that the Psychology Department needs to evaluate the data results, the methodologies utilized, and develop a new strategy to assess this course component of the program. The following plan of action for improvement is recommended:
1. Develop a survey instrument to utilize with the faculty who were randomly selected to participate in this study.
2. Improve the methodology utilized in this study.
3. Survey all faculty teaching PSYC.2301 to determine if they are aware of the “Core Topics” within this course.
4. Discuss the need to develop additional specific outcomes related to assessing the attainment of excellence within this course.
Dr. Dan Dydek and Dr. María G.Cisneros-Solís will present the results and proposed action plan at the next task force meeting. A recommendation will be forwarded to create an assessment team within the Psychology Department that will be responsible to implement this action plan during the academic year of 2003-2004.
Identification
of Impact
This component of the assessment study will be determined next academic year (2003-2004) after the implementation of the improvement action plan.
Appendix
PSYC 2301
Introduction to
Psychology
Assessment
Quiz
1. Results of a research study produced a correlation coefficient of +0.89 for the
variables of owning a portable video game player and being aggressive at home.
Which of the following is a valid conclusion?
A.
Portable
video games cause aggression.
B.
Aggressive
children like to buy portable video game players.
C.
Parents
of aggressive children tend to ignore their toy preferences.
D.
The
high correlation does not prove anything caused anything else.
2. Your high school principal sends you a
questionnaire to complete about many of
your high school experiences. The
instructions tell you to be as honest as possible
and that the questionnaire has been
sent out to hundreds of other graduates. In all
likelihood, your principal is using
which of the following research methods?
A. survey.
B. experiment.
C. naturalistic observation.
D. case study.
D.
“Open
to novelty and challenge.”
4. Five
year old Sandy has not wet her bed for over two years. However, she starts bed-
wetting again soon after her little brother is born. Sandy’s behavior is most likely an
example of the following defense mechanism:
A.
reaction
formation
B.
regression
C.
projection
D.
sublimation
E.
repression
6. Which of
the following behaviors is most likely to have been acquired through the
process
of classical conditioning?
A.
Cleaning your room so your parents will pay you
B.
falling off your bicycle and learning to try again by
observing your friend
C.
wincing when you hear the dentist’s drill
D.
playing tennis
7. When the civil rights heroine of the
sixties, Rosa Parks, refused to give up her seat
for a white man on the Montgomery,
Alabama city bus, at what level of Moral
Development was she functioning,
according to Kohlberg?
A.
Preconventional
Level
B.
Conventional
Level
C.
Post-Conventional
Level
D.
Female
Assertiveness Level
E.
Heroic
Level
8. Innate ability is to learned skill as is to .
A.
Erikson;
Piaget
B.
nature;
nurture
C.
stability;
stages
D.
Harlow;
Piaget
10. Which neurotransmitter has been linked to schizophrenia
and Parkinson’s disease?
A.
epinephrine
B.
serotonin
C.
acetylcholine
D.
dopamine
11. Ferby lives next to the railyards. He has lived there for years and no longer
notices
the trains going by. He has experienced __________.
A.
the
rail yard blues
B.
Sensitization
C.
psychic
numbing
D.
Desensitization
E.
a
gradual mental deterioration
12. An example of long-term memory is
A. answering general
information questions like Who is the
Governor of
Texas?
B. observing three designs
for 10 seconds and then identifying the same three designs from another page
containing nine designs.
C. listening to the
following set of numbers, 39865217654,
and then
repeating the list in reverse order.
D. calling directory
assistance from a pay telephone and remembering the
number long enough to make the call.
13. What's wrong with being a "Type A
Personality?"
A.
Nobody
likes you.
B.
You
like nobody.
C.
Life
is a drag.
D.
You
may die young.
E.
Nothing's
wrong with it. It's cool.
14. Stress
tends to lower the functioning of our ______system thus making us more prone to
disease and infections.
A.
digestive
B.
immune.
C.
circulatory.
D.
emotional.
15. Mike Hargrove, a former first baseman for the Texas Rangers,
always went through
an extensive ritual before being ready to bat (i.e., check
fit of each batting glove
twice, knock the cleats on each shoe several times with the
bat, take four or five
practice swings, etc.).
What type of behavior is this?
A. phobic
B. histrionic
C. narcissistic
D. obsessive-compulsive
16. Because it occurs so frequently and
because almost everyone has experienced
elements of the disorder at some time
in life, _________ is characterized as the
“common cold of psychopathology.”
A.
dissociative
identity disorder
B.
anxiety
disorder
C.
depression
D.
schizophrenia
17. If someone changes his/her attitudes or
overt behavior to adhere to social norms,
what is that person doing?
A. Obeying authority
B.
Being
a social animal
C.
Experiencing
cognitive dissonance
D.
Conforming
E.
All
of the above
18. What is the unfounded belief that a person,
or group, on the basis of assumed
racial, ethnic, sexual, or other
features will possess negative characteristics or
perform inadequately?
A.
Prejudice
B.
Discrimination
C.
Stereotyping
D.
Profiling
E.
Bigotry