The Building Blocks of Social Scientific Research Measuremen
Measurement involves deciding how to measure the presence, absence, or number of concepts in a research project. Reliability and validity of measures are key concerns.
A reliable measure yields a consistent, stable result as long as the concept being measured remains unchanged. Measurement strategies that rely on memories, for example, may be quite unreliable because the ability to remember specific information may vary depending on when the measurement is made and whether distractions are present.
Valid measures correspond well with the meaning of the concept being measured. Researchers often develop rather elaborate schemes to measure complex concepts.
Level of measurement is an important aspect of a measurement scheme. There are four levels of measurement. From lowest to highest, these levels are as follows: nominal, ordinal, interval, and ratio. Choosing the appropriate statistics for the analysis of data depends on knowing the level of measurement of your variables. Frequently a variable can be measured using a variety of schemes. Choosing the scheme that uses the highest level of measurement possible provides the most information and is the most precise measure of a concept. Researchers frequently recode data, thus changing the level of measurement of a variable.
Helpful Hints
Recoding Data There are two strategies for recoding data to combine or collapse categories of a measure:
1. Theoretical. Choose categories that are meaningfully distinct, where theory would tell you that the differences between the categories are important or where you can see that there are distinct clusters of scores or values. For example, when combining actual household income amounts into income levels, a researcher might consider what the official poverty level is and group all households with incomes below that level into the lowest income group.
2. Equally Sized Categories. Choose categories so that each category has roughly an equal number of cases. In addition, limit the number of categories so that each category has at least ten cases.
1/23/2019 VitalSource: Working with Political Science Research Methods
http://bookshelf.vitalsource.com/#/books/9781506306728/cfi/6/26!/4/2/4/2@0:0 2/13
PRINTED BY: tmomen@gmail.com. Printing is for personal, private use only. No part of this book may be reproduced or transmitted without publisher’s prior permission. Violators will be prosecuted.
Exercise 5–1. What is the level of measurement of the following measures? If you think there could be more than one level of measurement, explain your answer.
a. Type of government system (authoritarian, communist, democracy, monarchy, other)
___________________________________________________________________________________________
___________________________________________________________________________________________
b. Number of vetoes issued by each president of the United States
___________________________________________________________________________________________
___________________________________________________________________________________________
c. Race (Asian, Black, Hispanic, White, other)
___________________________________________________________________________________________
___________________________________________________________________________________________
d. Hours spent on social media per day (0-1, 2-3, 4 or more)
___________________________________________________________________________________________
___________________________________________________________________________________________
e. Literacy rate in each country (percentage literate)
___________________________________________________________________________________________
___________________________________________________________________________________________
f. State uses the death penalty (yes, no)
___________________________________________________________________________________________
___________________________________________________________________________________________
g. Number of witnesses in a Senate committee hearing
___________________________________________________________________________________________
___________________________________________________________________________________________
h. Political ideology (very conservative, conservative, moderate, liberal, very liberal)
___________________________________________________________________________________________
___________________________________________________________________________________________
1/23/2019 VitalSource: Working with Political Science Research Methods
http://bookshelf.vitalsource.com/#/books/9781506306728/cfi/6/26!/4/2/4/2@0:0 3/13
i. PRINTED BY: tmomen@gmail.com. Printing is for personal, private use only. No part of this book may be reproduced or transmitted without publisher’s prior permission. Violators will be prosecuted.
Party identification (Democrat, Republican, Independent, other, none)
___________________________________________________________________________________________
___________________________________________________________________________________________
j. Number of border countries
___________________________________________________________________________________________
___________________________________________________________________________________________
k. Member of U.S. Chamber of Commerce (yes, no)
___________________________________________________________________________________________
___________________________________________________________________________________________
l. Year first elected to public office
___________________________________________________________________________________________
___________________________________________________________________________________________
m. Primary policy objective of military intervention (foreign policy restraint, humanitarian intervention, internal political change)
___________________________________________________________________________________________
___________________________________________________________________________________________
n. Tone of campaign commercial (positive, mixed, negative)
___________________________________________________________________________________________
___________________________________________________________________________________________
o. Year in college (freshman, sophomore, junior, senior)
___________________________________________________________________________________________
___________________________________________________________________________________________
p. Health care system (government-managed health care, health care coverage required by law, private health care system)
___________________________________________________________________________________________
___________________________________________________________________________________________
1/23/2019 VitalSource: Working with Political Science Research Methods
http://bookshelf.vitalsource.com/#/books/9781506306728/cfi/6/26!/4/2/4/2@0:0 4/13
PRINTED BY: tmomen@gmail.com. Printing is for personal, private use only. No part of this book may be reproduced or transmitted without publisher’s prior permission. Violators will be prosecuted.
Exercise 5–2. Levels of measurement are important because they serve as a way to think about both the amount of information available in a measure and the mathematical properties of the measure. In this exercise you are going to consider the amount of information available in variables that measure the same concept with different levels of measurement. For each of the variables below, identify the level of measurement. Second, explain why one variable provides more information than the other. Finally, why might you prefer to use one measure over the other? Why is capturing more information important?
Concept: Education
Variable #1:
What is your highest completed level of education?
1. No formal education 2. Elementary school 3. Middle school 4. High school 5. College 6. Advanced degree
Variable #2:
How many years of formal education have you completed?
___________________________________________________________________________________________
___________________________________________________________________________________________
___________________________________________________________________________________________
___________________________________________________________________________________________
Exercise 5–3. Herrmann, Tetlock, and Visser define the disposition of military assertiveness as “the inclination toward different methods of defending American interests abroad, in particular, whether a person prefers more militant and assertive strategies or more accommodative and cooperative approaches.”1 To measure military assertiveness, they used ten items. For the first eight items, they asked respondents to indicate whether they strongly agreed, agreed, neither agreed nor disagreed, disagreed, or strongly disagreed with the statement.
Which of the following items do you think are the most valid measures of the concept of military assertiveness and why? Which ones do you have trouble relating to the concept and why? What kind of validity (face or construct) do you think the items exhibit?
1. The best way to ensure world peace is through American military strength. 2. The use of military force only makes problems worse. 3. Rather than simply reacting to our enemies, it’s better for us to strike first. 4. Generally, the more influence America has with other nations, the better off they are. 5. People can be divided into two distinct classes: the weak and the strong.
1/23/2019 VitalSource: Working with Political Science Research Methods
http://bookshelf.vitalsource.com/#/books/9781506306728/cfi/6/26!/4/2/4/2@0:0 5/13
PRINTED BY: tmomen@gmail.com. Printing is for personal, private use only. No part of this book may be reproduced or transmitted without publisher’s prior permission. Violators will be prosecuted.
6. The facts on crime, sexual immorality, and the recent public disorders all show that we have to crack down harder on troublemakers if we are going to save our moral standards and preserve law and order.
7. Obedience and respect for authority are the most important virtues children should learn. 8. Although at times I may not agree with the government, my commitment to the United States always remains
strong. 9. When you see the American flag flying, does it make you feel extremely good, somewhat good, or not very
good? 10. How important is military defense spending to you personally: very important, important, or not at all
important?
Most valid measures of the concept of military assertiveness:
___________________________________________________________________________________________
___________________________________________________________________________________________
___________________________________________________________________________________________
___________________________________________________________________________________________
Worst “fit” for concept:
___________________________________________________________________________________________
___________________________________________________________________________________________
___________________________________________________________________________________________
Kind of validity:
___________________________________________________________________________________________
___________________________________________________________________________________________
___________________________________________________________________________________________
___________________________________________________________________________________________
Exercise 5–4. Suppose you think that moral values are theoretically important in explaining voting behavior. Before you can write your theory or test your hypotheses involving moral values, you must conceptualize and operationalize the concept. In the space following, conceptualize (define the term) and operationalize (decide how you will record the quantitative variable) moral values.
Conceptualization: ________________________________________________________________________
___________________________________________________________________________________________
___________________________________________________________________________________________
1/23/2019 VitalSource: Working with Political Science Research Methods
http://bookshelf.vitalsource.com/#/books/9781506306728/cfi/6/26!/4/2/4/2@0:0 6/13
PRINTED BY: tmomen@gmail.com. Printing is for personal, private use only. No part of this book may be reproduced or transmitted without publisher’s prior permission. Violators will be prosecuted.
Operationalization: _________________________________________________________________________
___________________________________________________________________________________________
___________________________________________________________________________________________
___________________________________________________________________________________________
___________________________________________________________________________________________
___________________________________________________________________________________________
Exercise 5–5. Operationalization is deciding how to record empirical observations of the occurrence of an attribute or a behavior using numerals or scores. In other words, it is deciding how to move from defined concept to quantifiable variable. In this exercise you are going to consider the challenges involved in quantifying both concrete and abstract concepts that are commonly used in political science research. You will find below a series of conceptualized terms. Your job is to explain how you would operationalize each term for use in a survey research project by creating the questions that would yield the appropriate variable for each concept. (Hint: Concrete terms are much easier to work with than abstract terms. Pay close attention to the abstract terms, such as ideology and efficacy.)
Example: Voter registration: Whether someone is currently registered to vote.
Answer: Ask each respondent to indicate whether he or she is currently registered to vote by asking, “Are you currently registered to vote in your state?” (1) Yes, I am registered to vote; (0) No, I am not registered to vote.
a. Gender: Male and female
___________________________________________________________________________________________
___________________________________________________________________________________________
___________________________________________________________________________________________
b. Household income: The amount of money earned by all members of a household in a year
___________________________________________________________________________________________
___________________________________________________________________________________________
___________________________________________________________________________________________
c. Race: The race each respondent most closely identifies with
___________________________________________________________________________________________
___________________________________________________________________________________________
___________________________________________________________________________________________
1/23/2019 VitalSource: Working with Political Science Research Methods
http://bookshelf.vitalsource.com/#/books/9781506306728/cfi/6/26!/4/2/4/2@0:0 7/13
d. PRINTED BY: tmomen@gmail.com. Printing is for personal, private use only. No part of this book may be reproduced or transmitted without publisher’s prior permission. Violators will be prosecuted.
Ideology: A set of beliefs and ideas, including one’s moral code and worldview. The most important issues and ideas involve how the government should address those unable to provide food, health care, and housing for themselves and their children. The extent to which the government should extend services to support those in need in these areas makes up the worldview.
___________________________________________________________________________________________
___________________________________________________________________________________________
___________________________________________________________________________________________
e. Political efficacy: The belief that one’s political action will have a meaningful effect. In particular I define political action as interpersonal communication with elected officials.
___________________________________________________________________________________________
___________________________________________________________________________________________
___________________________________________________________________________________________
Exercise 5–6. Table 5–1 contains a frequency distribution of senators’ scores on the American Federation of Labor– Congress of Industrial Organizations (AFL–CIO) rating system for 2005. The left-hand column shows the actual scores given to senators; the columns to the right show how many senators received the scores. Suppose you wanted to code these data into an ordinal-level measure with two categories. What are two ways this could be done? Give the range of scores that would fall into the categories in your ordinal-level measures.
First ordinal-level measure: __________________________________________________________________
___________________________________________________________________________________________
___________________________________________________________________________________________
___________________________________________________________________________________________
Second ordinal-level measure: _______________________________________________________________
___________________________________________________________________________________________
___________________________________________________________________________________________
___________________________________________________________________________________________
1/23/2019 VitalSource: Working with Political Science Research Methods
http://bookshelf.vitalsource.com/#/books/9781506306728/cfi/6/26!/4/2/4/2@0:0 8/13
PRINTED BY: tmomen@gmail.com. Printing is for personal, private use only. No part of this book may be reproduced or transmitted without publisher’s prior permission. Violators will be prosecuted.
Table 5–1 AFL–CIO 2005 Senators’ Rating Scores
Score Frequency Percentage Cumulative Percentage
0 1 1.0 1.0
7 16 16.0 17.0
8 1 1.0 18.0
14 19 19.0 37.0
15 2 2.0 39.0
17 1 1.0 40.0
21 6 6.0 46.0
23 1 1.0 47.0
29 3 3.0 50.0
46 1 1.0 51.0
50 1 1.0 52.0
57 3 3.0 55.0
64 1 1.0 56.0
69 1 1.0 57.0
71 3 3.0 60.0
77 2 2.0 62.0
79 10 10.0 72.0
85 1 1.0 73.0
86 7 7.0 80.0
92 3 3.0 83.0
93 8 8.0 91.0
100 9 9.0 100.0
1/23/2019 VitalSource: Working with Political Science Research Methods
http://bookshelf.vitalsource.com/#/books/9781506306728/cfi/6/26!/4/2/4/2@0:0 9/13
Score Frequency Percentage Cumulative Percentage
Total 100 100.0 Source: American Federation of Labor–Congress of Industrial Organizations (AFL–CIO), 2005.
Exercise 5–7. Table 5–2 shows the distribution of the average index scores of each state’s delegation to the U.S. House of Representatives on the League of Conservation Voters (LCV) index for 2006. The index ranges from 0 to 100 and represents the percentage of times that a member voted in favor of the LCV position on selected issues. Suppose that you wanted to group the average state delegation scores into four categories for a new variable called “Support for LCV.” What range of values would be included in each of the categories? Justify your answer.
___________________________________________________________________________________________
___________________________________________________________________________________________
___________________________________________________________________________________________
___________________________________________________________________________________________
1/23/2019 VitalSource: Working with Political Science Research Methods
http://bookshelf.vitalsource.com/#/books/9781506306728/cfi/6/26!/4/2/4/2@0:0 10/13
PRINTED BY: tmomen@gmail.com. Printing is for personal, private use only. No part of this book may be reproduced or transmitted without publisher’s prior permission. Violators will be prosecuted.
Table 5–2 League of Conservation Voters 2006 State Delegation Averages for the House
Score Frequency Percentage Cumulative Percentage
0 3 6.0 6.0
4 1 2.0 8.0
5 1 2.0 10.0
9 1 2.0 12.0
14 1 2.0 14.0
19 1 2.0 16.0
20 2 4.0 20.0
22 2 4.0 24.0
25 1 2.0 26.0
26 1 2.0 28.0
27 2 4.0 32.0
32 1 2.0 34.0
36 1 2.0 36.0
37 1 2.0 38.0
38 2 4.0 42.0
39 4 8.0 50.0
40 1 2.0 52.0
42 1 2.0 54.0
45 1 2.0 56.0
47 1 2.0 58.0
50 2 4.0 62.0
1/23/2019 VitalSource: Working with Political Science Research Methods
http://bookshelf.vitalsource.com/#/books/9781506306728/cfi/6/26!/4/2/4/2@0:0 11/13
Score Frequency Percentage Cumulative Percentage
52 1 2.0 64.0
54 1 2.0 66.0
61 1 2.0 68.0
62 1 2.0 70.0
67 3 6.0 76.0
68 1 2.0 78.0
75 1 2.0 80.0
77 1 2.0 82.0
78 2 4.0 86.0
83 1 2.0 88.0
85 1 2.0 90.0
88 2 4.0 94.0
89 1 2.0 96.0
99 1 2.0 98.0
100 1 2.0 100.0
Total 100 100.0 Source: League of Conservation Voters index, 2006.
1/23/2019 VitalSource: Working with Political Science Research Methods
http://bookshelf.vitalsource.com/#/books/9781506306728/cfi/6/26!/4/2/4/2@0:0 12/13
PRINTED BY: tmomen@gmail.com. Printing is for personal, private use only. No part of this book may be reproduced or transmitted without publisher’s prior permission. Violators will be prosecuted.
Exercise 5–8. Below you will find a series of hypotheses. For each hypothesis identify the variables you would need to test the hypothesis and explain how you could measure each variable. When explaining your measurement strategy, be careful to consider validity and reliability.
a. Small business owners are more likely to support tax cuts than other voters.
___________________________________________________________________________________________
___________________________________________________________________________________________
___________________________________________________________________________________________
___________________________________________________________________________________________
___________________________________________________________________________________________
b. The availability of government-subsidized child care causes household income to rise.
___________________________________________________________________________________________
___________________________________________________________________________________________
___________________________________________________________________________________________
___________________________________________________________________________________________
___________________________________________________________________________________________
c. An increase in the number of nongovernmental organizations operating in an authoritarian state increases the rate at which the state democratizes.
___________________________________________________________________________________________
___________________________________________________________________________________________
___________________________________________________________________________________________
___________________________________________________________________________________________
___________________________________________________________________________________________
d. Access to clean drinking water causes life expectancy to increase.
___________________________________________________________________________________________
___________________________________________________________________________________________
___________________________________________________________________________________________
___________________________________________________________________________________________
___________________________________________________________________________________________
___________________________________________________________________________________________
___________________________________________________________________________________________
1/23/2019 VitalSource: Working with Political Science Research Methods
http://bookshelf.vitalsource.com/#/books/9781506306728/cfi/6/26!/4/2/4/2@0:0 13/13
1 Richard K. Herrmann, Philip E. Tetlock, and Penny S. Visser, “Mass Public Decisions to Go to War: A Cognitive- Interactionist Framework,” American Political Science Review 93 (September 1999): 554.
Use the following coupon code :
ULTIMATE