Quantitative Methods - Methods Section - example

(As a background info: this is the methods section of the example I showed in class - working with the research question as to whether membership in church or religious organizations impact activism and membership in other organizations).

This paper focuses on Brazil because CEBs were more active and numerous in this country and as the Catholics constitute the majority of the population in terms of religious denomination (69%). However, the study aims to test the hypothesis for all religious denominations and not for the Catholics only. As it was mentioned in the previous section the period when the CEBs were more influential and politically significant was the end of the 1970s and the early 1980s. The second wave (1990-1991) of the WVS will be employed in this paper since Brazil was not included in its first wave (1981). It does not make much sense to analyze the third wave (1995) either since, as we have stated above, the 1990s saw the relative decline of the importance of the Catholic Church and the rise of Protestantism in Latin America.

3.1. Sampling and Data
The WVS was initiated in 1981 to study the values and attitudes of people belonging to different economic, educational, and cultural backgrounds. This cross-national survey explores whether mass belief systems are changing throughout the world. The surveys are administered using stratified, multistage random sampling and covers adult citizens aged 18 and older. The samples are selected in two stages, first of which is a random selection of sampling locations. This is done in order to ensure all types of locations are represented. Among these sampling locations a random selection of individuals is drawn. Multi-stage random sampling first selects the household randomly from these locations, and then chooses the individuals in the household by using either a grid to select the person to be interviewed from a listing of all those in the household or in other countries through quota sampling with quotas assigned on the basis of sex, age and occupation, using census data as a guide to the distribution of each group in the population. The interviews are structured and are carried out face-to-face at home. The advantages of face-to-face interviews are high response rates, collection of supplementary information and control of the interview situation (Frankfort-Nachmias and Nachmias, 2000). The interviews were carefully worded/translated in order to facilitate the correct understanding of questions. The surveys from most low-income countries undersample the illiterate portion of the public and oversample the urban areas and the more educated strata (Inglehart, 1994). Since the present data set is also from a developing country it is weighted by using the weight variable (V236 in the WVS Codebook) to correct for these errors.

The WVS data for Brazil have been gathered by the Instituto Gallup de Opiniao Publica in Sao Paulo. The survey was administered between October 1991 and January 1992. The size of the sample is 1782. There is no information concerning the non-response rate in this wave of the survey. Therefore, it is not possible to determine the bias caused by this non-sampling error. The basic demographic characteristics of the Brazilian sample, under the categories of age, sex and education, are provided in Table 1.


3.2. The Dependent Variable: Activism
As it was stated earlier, this study will test whether membership in a church or a religious organization affects the level of mobilization of Brazilians. For the purposes of this paper mobilization means participation in non-violent acts organized by civil society groups and is tapped using two indices.[1] The variables that constitute the indices and how they are constructed will be outlined below. Their characteristics in terms of frequency distributions and descriptive statistics will also be provided. Subsequently their validity and reliability will be examined.

The first index is the variable activism and is constructed using four questions that were put forth to measure the degree of social activism of the interviewed individuals. The original set of questions identifies five types of activism – signing a petition, joining in boycotts, attending lawful demonstrations, joining unofficial strikes and occupying buildings and factories – but the last item is not relevant for this study and will be omitted. Interviewees were asked to respond to each one by stating whether they have done it (1), might do it (2) or would never do it (3). The WVS is designed such that the validity of each measurement item is maximized. However, a brief discussion of validity is necessary. Validity refers to whether an instrument measures what it is supposed to measure (Frankfort-Nachmias and Nachmias, 2000).

It is necessary to examine different types of validity that are relevant for the measurement process of this paper. For a measurement to be content valid, it should cover all attributes of the concept to be measured. One sub-category of construct validity is face validity and it refers to the subjective evaluation of the researcher – through comparing the survey questions with already existing surveys, for example, or through consulting specialists in the area. The WVS has been administered in different countries over three waves and there was ample possibility to check the face validity of the measures. The second category is sampling validity and it refers to whether the “full domain of content that is relevant to the particular measurement situation (Carmines and Zeller, 1979)” is specified. It can be argued that the variables making up our index (activism) could have been more valid had there been a question about joining organizations of self-help and participation in neighborhood associations. Some of the variables – such as joining a boycott or signing a petition – may be more related to the Western European (where the surveys originated) context rather than the Latin American one. Hence there is the possibility of a mismatch of repertoires of action and the range of questions concerning activism for the case of Brazil. Frankfort-Nachmias and Nachmias (2000) state that construct validity is achieved when a researcher relates the measuring instrument to the general theoretical framework. Variables used in order to construct the index are derived from the theoretical contributions to the field of social mobilization and activism. However, the same gap between the Latin American mobilization and Western European counterpart makes the index less capable of capturing all the relevant features of social activism in Brazil.

The index adds the values for the answers of each individual for the four types of activism, i.e. the four items. The new variable ranges between 4 and 12, where an individual who scores 4 has done all types of activism at least once whereas someone who scores 12 would never do any of them. This new scale was tested for its internal consistency or reliability[2] using the reliability analysis incorporated into the SPSS. The value of alpha – which signifies the proportion of variation in the subjects that is explained by the new scale – generated by the program (0.70) indicates that the index is reliable.[3] Subsequently, the index was re-coded in order to obtain a new scale variable where the 4-7 range becomes 1 and means that the person is active; 8-10 range becomes 2 and means that the person is potentially active; and 10-12 range becomes 3 and means that the person is not active. The characteristics of the activism variable are shown in Graph 1.

Graph. 1 Frequency Distribution of Activism

3.3 The Dependent Variable: Membership
The second index, membership, is constructed using the questions that tap the level of membership in voluntary organizations other than the church. Types of voluntary organizations used in order to calculate the index are labor unions (which may not always be voluntary), political parties, environmental organizations, professional associations, charitable organizations and any other voluntary organization.[4] The question that tapped these variables asked the respondents whether they are active members (1), inactive members (2) or not a member (3) of these organizations.

The concerns related to validity for the items of the previous index are not relevant here due to the existence of the residual item – any other voluntary organization. The index is constructed by adding the items. The new variable, which ranges between 6 and 18, was re-coded into a nominal variable where some membership, namely 6-16, was coded as 1 and no significant membership, namely 17-18, was coded as 2. However, the reliability analysis of membership index reports an alpha of 0.45, which indicates that the index is unreliable. Therefore the results obtained using this variable should be interpreted accordingly. The frequency distribution of the membership variable is given in Graph 2.

Graph 2. Frequency Distribution of Membership



3.4 Independent Variables

The independent variables used in order to explain the variation in activism and membership and the descriptions of these variables are given below.


3.4.1. Membership in Church or a Religious Organization
This variable is the main independent variable and is measured using the question that asks the respondents whether they are active, inactive members of or do not belong to a church or a religious organization.[5] The code for active member is 1 whereas that of a non-member is 3. This variable is moderately correlated with attendance in religious services (see Table 2) and hence the measure we will be using for this study as an independent variable has predictive validity. Namely, the church and religious organization membership is an accurate measurement instrument and taps the relevant information.


However, the exploration of the characteristics of this variable demonstrates that only 12.8 % of the sample is actively member of a church or religious organization (see Table 3). This skewed distribution will not create a problem in terms of the size of the sub-category of active or inactive membership during the analysis phase since Nactive=228 and Ninactive=168 are still large samples (as a rule of thumb N>30 is considered large).

Table 3. Frequency Distribution of Membership in Church or Religious Organization



3.5. Control Variables
General descriptive characteristics of the control variables – gender, age, education and size of town – are provided in Table 4.

      Table 4. Descriptive Statistics


3.5.1. Gender
The gender of the respondent is coded 1 in the case of a male interviewee and 2 in the case of a female. The gender is also a very important aspect for the study of mobilization and activism in Latin America. However, due to the concerns articulated in the previous sub-section the effect of gender may not be as significant as those reported by the microanalyses conducted in Latin America.

3.5.2. Age
The age variable is important to include as a control variable because there can be significant differences among age groups in terms of activism and membership in voluntary organizations. This variable was measured using a question that asks how old the person is. However, in the Brazilian case it was coded within intervals as follows: 18-21, 22-27, 28-35, 36-45 and 46+. The frequency distribution of this ordinal variable is given in Graph 3.



3.5.3. Education
Education was tapped using the following question: What is the highest educational level that you have attained? For the case of Brazil the categories were as follows: 1- Illiterate/never went to school; 2- Primary school; 3- Secondary school and 4- Higher education. The frequency distribution of this variable is given in Graph 4.

Education is an important variable because research on social movements and mobilization in Europe tends to find higher levels of education being correlated with higher levels of participation. However, in the case of Brazil had the activism variable included many of the activism associated with people of low-education – such as the neighborhood organizations or self-help communities – we could expect an opposite relationship. The reason for this is the vibrant nature of these grassroots organizations in the case of Latin America and their typical members being women and not very educated. However, since the type of activism captured by the questions of the WVS is closer to the European types it is still important to analyze whether education is a factor which affects the level of activism and membership in voluntary organizations.

3.5.4. Community Size
Community size, or the size of town, where the individual lives can affect the level of mobilization in various ways. The categories used for coding the Brazilian sample and the frequency distribution are given in Table 5. As it was mentioned before, large cities are oversampled in the survey but the use of weights in the analysis will prevent this from biasing the analysis.
       Table 5. Coding and Frequency of Size of Town (N=1782)


3.5.5. Employment Status
The employment status is coded as follows: 1- employed; 2- retired; 3- housewife; 4- student; 5 unemployed. The intervening effect of the employment status may be of relevance for the purposes of this study. It is possible to conjecture that the effect of church or a religious organization membership on activism or membership is dependent on the employment status of the person. Those who are employed will have more opportunity to join demonstrations, strikes and boycotts and therefore their activism level may be higher in comparison to those who are retired or unemployed.


[1] An index is a composite variable constructed using several variables. The variables that constitute an index are called items (Frankfort-Nachmias and Nachmias, 2000).

[2] Reliability refers to the extent to which a measuring procedure yields the same result on repeated trials. In order to measure the internal consistency or reliability a measure called the Cronbach’s alpha is used (Carmines and Zeller, 1979).

[3] Frankfort-Nachmias and Nachmias (2000) state that in order to claim a scale is reliable the alpha value should be greater than or equal to 0.70 whereas the limit set by Cramer (1998) is 0.80.

[4] This variable excludes all sorts of religious or church organizations.
[5] This variable will sometimes be referred to as church membership for the sake of simplicity.