PROGRAM EFFECTIVENESS: STUDENT LEARNING OUTCOMES
ULEAD
ASSESSMENT OF PSYC 2301
Dan
B. Dydek, Ph.D.
Associate
Professor of Psychology
María
Cisneros-Solís, Ph.D.
Professor
of Psychology
May
28, 2004
PROGRAM
EFFECTIVENESS: STUDENT LEARNING OUTCOMES
ULEAD
ASSESSMENT OF PSYC 2301 (FY2003-2004)
Purpose/Goal
The Department of Psychology at ACC
values excellence in the quality of our courses and delivery of instruction.
Continual improvement of instruction is, therefore, a major goal of this component
of our program effectiveness review process.
Last year the PSYC 2301 course was
assessed using an instrument designed collectively by the Psychology
Faculty. The report of that assessment
is available online at this URL: http://www.austincc.edu/behscitf/PSYC%202301%20Assessment%20Results.htm
In the Fall of 2003 as part of the
Faculty Development Day Discipline-Specific Workshops, a report of the
assessment was presented to the faculty and several faculty members presented their teaching methods as
they pertain to specific Core Objectives for PSYC 2301. In this manner the results and implications
of last year’s assessment were communicated to faculty as well as suggestions
for improving pedagogy.
This academic year (2003-2004) we have
again selected PSYC 2301, Introduction to Psychology. This year’s assessment is
a follow-up to last year’s study. This course is required for an Associate’s
and Bachelor’s Degree in Psychology, as well as in several other majors. It
transfers to most universities in Texas and the content knowledge and skills
covered within the course will enhance the students’ success in their chosen
careers and everyday life situations.
Learning Outcomes
The learning outcomes of PSYC 2301
incorporate a comprehensive overview of psychological principles and concepts,
which students can apply to their chosen careers and everyday life
situations. The specific core topics including
representative theories and research that need to be mastered by our students
include the following:
·
Research
Methods
·
Personality
·
Learning
·
Life-Span
Development
·
Physiological
Basis of Behavior
·
Cognition
(including Sensation, Perception & Memory)
·
Stress
(and intellectual, social, physical and emotional health)
·
Psychological
Disorders
·
Social
Psychology
Prior to the beginning of the fall, 2002 semester
the Psychology faculty within the Behavioral Sciences Task Force decided on the
following Specific Learning Outcomes and Assessment Criteria for PSYC 2301,
Introduction to Psychology:
Specific
Learning Outcomes:
Upon completion of the course the majority of
students will demonstrate the acquisition of knowledge and mastery of the “Core
Topics” for the Introduction to Psychology course (PSYC 2301).
Assessment
Criteria:
In a test given at the end of the course, 70% of
students quizzed on a multiple-choice post-test will answer correctly items
specific to each of the nine “Core Topics” in PSYC 2301.
Methodology
As a follow-up to last year’s study, in order to
compare the students’ performance on the Assessment Quiz when taught by an
instructor well versed in the Core Topics, five sections of PSYC 2301, all
taught by one lecturer were assessed. The
anonymous lecturer is very familiar with the Core Topics.
The
students answered the eighteen item multiple choice quiz (See Appendix) on
Scantron forms which were aggregated, scored, and item analyzed to determine
the percentage correct on each of two items relevant to each of the nine Core
Topics. 105 students participated in this year’s assessment study.
Results
Table 1.0 shows the number and
percentage of students who incorrectly (and correctly) answered each item on
the Assessment Quiz. Also shown is the
Mean % Correct for Each Core Topic, which was obtained by calculating the mean
of the percent correct scores on the two items representing each Core Topic. Figure 1.0 also shows the Mean
Percentage Correct for each Core Topic.
Table 1.0
|
Item # |
# missed |
# correct |
% correct |
% correct per topic |
|
1 |
50 |
55 |
52.4 |
Research
Methods |
|
2 |
12 |
93 |
88.6 |
70.5 |
|
3 |
12 |
93 |
88.6 |
Personality |
|
4 |
36 |
69 |
65.7 |
77.1 |
|
5 |
33 |
72 |
68.6 |
Learning |
|
6 |
44 |
61 |
58.1 |
63.3 |
|
7 |
34 |
71 |
67.6 |
Life-Span
Development |
|
8 |
22 |
83 |
79.0 |
73.3 |
|
9 |
56 |
49 |
46.7 |
Physiological Basis of Behavior |
|
10 |
34 |
71 |
67.6 |
57.1 |
|
11 |
6 |
99 |
94.3 |
Cognition (inc.
Sensation, Perception & Memory) |
|
12 |
31 |
74 |
70.5 |
82.4 |
|
13 |
38 |
67 |
63.8 |
Stress and Health |
|
14 |
2 |
103 |
98.1 |
81.0 |
|
15 |
13 |
92 |
87.6 |
Psychological Disorders |
|
16 |
16 |
89 |
84.8 |
86.2 |
|
17 |
23 |
82 |
78.1 |
Social Psychology |
|
18 |
43 |
62 |
59.0 |
68.6 |
Figure 1.0
These data indicate that the Assessment Criteria
pre-established by the Task Force were attained for six of the nine “Core
Topics,” i.e. Psych. Disorders, Cognition, Personality, Stress, Research
Methods, and Life-Span, with more than 70% of students answering relevant items
correctly in each topic. The three Core
Topics in which degree of mastery fell
below 70% mastery level were Physiology & Behavior, Learning and Social
Psychology, which scored 57.1%, 63.3% and 68.6 % respectively.
Table 2.0 shows the comparison between the 2003 data
and the 2004 data in terms of % of students answering correctly (on the
average) for each Core Topic.
|
Core Topic |
2003 (%) |
|
|
|
Psychological Disorders |
78.4 |
86.2 |
|
|
Cognition (including
Sensation, Perception & Memory) |
77.9 |
82.4 |
|
|
Personality |
76.4 |
77.1 |
|
|
Physiological Basis of
Behavior |
67.0 |
57.1 |
|
|
Stress (and intellectual,
social, physical and emotional health) |
62.4 |
81 |
|
|
Research Methods |
62.4 |
70.5 |
|
|
Life-Span Development |
56.6 |
73.3 |
|
|
Learning |
51.7 |
63.3 |
|
|
Social Psychology |
33.6 |
68.6 |
|
|
Grand Means: |
62.9 |
73.3 |
Table 2.0
Interpretation of Results
When the Grand Means of the percentage of students
answering correctly on the nine core topics were compared by means of a
Two-Sample t Test, the 2004 Grand Mean of 73.3% is significantly higher than
the 2003 Grand Mean of 62.9% (p < .05).
Unfortunately, this methodology does not allow one to infer what might
be the cause of this difference, although it is certain that the 2004 classes
were taught by an instructor well versed in the Core Topics.
With a 2004 Grand Mean of 73.3% of students
answering correctly items on the Core Topics, there is room for some
encouragement and also for some improvement.
Specifically, the aforementioned topics of Learning, Social Psychology
and Physiology, which were below the 70% criterion, need to be improved. On the other hand, the Grand Mean of 73.3 %
and six of the nine topics scoring above 70% indicates that this level of
mastery is attainable.
There is no way to ascertain the causes
of these measured outcomes based upon these data. All one can reliably conclude is that the
criterion of 70% of students achieving knowledge and mastery on each of the nine Core Topics has once
again not been demonstrated with this methodology. A reasonable inference might be that continued efforts to improve
our departmental effectiveness in PSYC 2301 would not be contraindicated. That conclusion is certainly compatible with
our departmental goal of continual improvement in all of our course offerings.
Implementation of Improvements
These results lead to the conclusion that the criterion of 70% of students achieving knowledge and mastery on each of the nine Core Topics has not been demonstrated with the methodology used in this assessment study. It appears that the Psychology faculty needs to evaluate the results, the methodologies utilized, and develop a new strategy to assess this course component of the program. The following plan of action for improvement is recommended:
1. Improve the methodology utilized in this study.
2. Survey all faculty teaching PSYC.2301 to determine if they are aware of the “Core Topics” within this course.
3. Discuss the need to develop additional specific outcomes related to assessing the attainment of excellence within this course.
Identification
of Impact
This component of the assessment study will be determined next academic year (2004-2005) after the implementation of the improvement action plan.
Appendix
PSYC 2301
Introduction
to Psychology
Assessment
Quiz
Questions Concerning Research Methods
1. Results of a research study produced a correlation coefficient of +0.89 for the
variables of owning a portable video game player and being aggressive at home.
Which of the following is a valid conclusion?
A.
Portable
video games cause aggression.
B.
Aggressive
children like to buy portable video game players.
C.
Parents
of aggressive children tend to ignore their toy preferences.
D.
The
high correlation does not prove anything caused anything else.
2. Your high school principal sends you a
questionnaire to complete about many of
your high school experiences. The
instructions tell you to be as honest as possible
and that the questionnaire has been
sent out to hundreds of other graduates. In all
likelihood, your principal is using
which of the following research methods?
A. survey.
B. experiment.
C. naturalistic observation.
D. case study.
Questions
Concerning Personality
D.
“Open
to novelty and challenge.”
4. Five year
old Sandy has not wet her bed for over two years. However, she starts bed-
wetting again soon after her little brother is born. Sandy’s behavior is most likely an
example of the following defense mechanism:
A.
reaction
formation
B.
regression
C.
projection
D.
sublimation
E.
repression
Questions Concerning
Learning
6. Which of
the following behaviors is most likely to have been acquired through the
process
of classical conditioning?
A.
Cleaning your room so your parents will pay you
B.
falling off your bicycle and learning to try again by
observing your friend
C.
wincing when you hear the dentist’s drill
D.
playing tennis
Questions Concerning
Lifespan Development
7. When the civil rights heroine of the
sixties, Rosa Parks, refused to give up her seat
for a white man on the Montgomery,
Alabama city bus, at what level of Moral
Development was she functioning,
according to Kohlberg?
A.
Preconventional
Level
B.
Conventional
Level
C.
Post-Conventional
Level
D.
Female
Assertiveness Level
E.
Heroic
Level
8. Innate ability is to learned skill as is to .
A.
Erikson;
Piaget
B.
nature;
nurture
C.
stability;
stages
D.
Harlow;
Piaget
Questions
Concerning Physiological Basis of Behavior
10. Which neurotransmitter has been linked to
schizophrenia and Parkinson’s disease?
A.
epinephrine
B.
serotonin
C.
acetylcholine
D.
dopamine
Questions
Concerning Cognition (including Sensation, Perception, and Memory
11. Ferby lives next to the railyards. He has lived there for years and no longer
notices
the trains going by. He has experienced __________.
A.
the
rail yard blues
B.
Sensitization
C.
psychic
numbing
D.
Desensitization
E.
a
gradual mental deterioration
12. An example of long-term memory is
A. answering general
information questions like Who is the
Governor of
Texas?
B. observing three designs
for 10 seconds and then identifying the same three designs from another page
containing nine designs.
C. listening to the
following set of numbers, 39865217654,
and then
repeating the list in reverse order.
D. calling directory
assistance from a pay telephone and remembering the
number long enough to make the call.
Questions
Concerning Stress and Health
13. What's wrong with being a "Type A
Personality?"
A.
Nobody
likes you.
B.
You
like nobody.
C.
Life
is a drag.
D.
You
may die young.
E.
Nothing's
wrong with it. It's cool.
14. Stress
tends to lower the functioning of our ______system thus making us more prone to
disease and infections.
A.
digestive
B.
immune.
C.
circulatory.
D.
emotional.
Questions
Concerning Psychological Disorders
15. Mike Hargrove, a former first baseman for the Texas Rangers,
always went through
an extensive ritual before being ready to bat (i.e., check
fit of each batting glove
twice, knock the cleats on each shoe several times with the
bat, take four or five
practice swings, etc.).
What type of behavior is this?
A. phobic
B. histrionic
C. narcissistic
D. obsessive-compulsive
16. Because it occurs so frequently and
because almost everyone has experienced
elements of the disorder at some time
in life, _________ is characterized as the
“common cold of psychopathology.”
A.
dissociative
identity disorder
B.
anxiety
disorder
C.
depression
D.
schizophrenia
Questions
Concerning Social Psychology
17. If someone changes his/her attitudes or
overt behavior to adhere to social norms,
what is that person doing?
A. Obeying authority
B.
Being
a social animal
C.
Experiencing
cognitive dissonance
D.
Conforming
E.
All
of the above
18. What is the unfounded belief that a person,
or group, on the basis of assumed
racial, ethnic, sexual, or other
features will possess negative characteristics or
perform inadequately?
A.
Prejudice
B.
Discrimination
C.
Stereotyping
D.
Profiling
E.
Bigotry