CIS - Centro de Investigaciones Sociológicas

Methodology

INTRODUCTION

The barometers of the Centro de Investigaciones Sociológicas (CIS) are its monthly opinion surveys, which monitor the opinions and attitudes of the Spanish society regarding current events and situations.

Most questions included in each month's survey deal with current events in the more strict sense (like the governing party's convention, the March 11 terrorist attack, etc.) or with a theme that is selected as the central one (like expectations for the incoming year, political culture, etc.). A small part of the questions are repeated over time, at different fixed intervals. Since the methodology used for these barometers is similar when it comes to their scope, universe, sampling procedure, sampling size, etc., the answers to these questions are comparable and give the corresponding temporal series.

In this way, in the barometers produce series of questions, monthly or quarterly, that deal with assessment of the current political situation and its future evolution, assessment of the current economical situation and its future evolution, assessment of the performance of the central government and of the main opposition party, the voting intention for general elections, voting predictions, and a scale of the ideological self-positioning of those questioned. These questions have been asked almost continuously and with very similar wording, since 1996 and therefore this date will be taken as a starting point to study and set up indicators, even if data from previous periods were available.

Although the answers to some of these questions constitute, in themselves, indicators of the evolution of certain variables, at times it is useful to try to consolidate a number of questions that refer to the same theme into a temporal series or an indicator that would sum up, in a single value, the whole of the information for any given moment in time. Also, in the case of multiple choice questions, it can be of interest to create a synthetic indicator.

The next section presents the methodologies for creating some of the best known indicators for social matters. Following this, the type of indicator chosen is presented and justified. In the final sections the methodologies for each series are set out in detail.

INDICATOR CREATION METHODOLOGIES

Index numbers are a traditional branch of statistics that deals with the creation of indicators or indices to compare economic and social events in time or space.

When it comes to events that can be measured using a quantitative variable, such as production volume, price level, etc., the literature on these matters offers a wide range of possibilities: simple indices, Laspeyres or Paasche compound indices, the ideal Fisher index, etc.

In the case of opinion surveys, the questions usually yield qualitative variables, with multiple choice answers. Therefore, the question:

yields a qualitative variable with five answer options. The results are presented, in general, as the percentage of answers obtained in each option.

Traditional indices are no use for this type of variable, so an alternative indicator that would synthesise all the information into a single value has to be created. This is usually done by analogy in the case of quantitative variables, calculating index numbers, answer balances, etc.

The following are some of the best known indicators created from international opinion surveys.

Consumer Confidence Index. This indicator has been being drawn up since 1967 for the Conference Board in the United States. The questionnaire is sent by mail to a representative sample of 5,000 families (of which an average of around 3,500 reply), which changes every month. It includes 5 questions:

1. Assessment of the current economic situation.
2. Expectations about the economic situation six months from now.
3. Assessment of the current employment situation.
4. Expectations about the employment situation six months from now.
5. Expectations about the total family income six months from now.

For each of these questions, there are three answer options: positive, negative or neutral.

For each question, the number of positive answers is divided by the sum of positive and negative to obtain the relative value, or ratio. This answer ratio is adjusted according to the season, and then divided by the average ratio for the year 1985 to obtain an index based on 1985 for each question.

The following indicators are obtained:

• Consumer Confidence Index. average of all 5 indices.
• Current situation index: average of the indices of questions 1 and 3.
• Expectations index: average of the indices of questions 2, 4 and 5.

Consumer Comfort Index. ABC News/Money magazine calculates this indicator based on a monthly telephone survey conducted nationally (in the United States) on about 1,000 adults, ever since 1985.

These are the three question it uses:

1. Would you describe the state of the nation’s economy these days as excellent, good, not so good, or poor?
2. Would you describe the state of your own personal finances these days as excellent, good, not so good, or poor?
3. Considering the cost of things today and your own personal finances, would you say now is an excellent time, a good time, a not so good time, or a poor time to buy the things you want and need?

Answer balances are calculated (the difference between positive and negative answers) for each question. Later, these 3 balances are added up and divided by 3, in order to obtain the Consumer Comfort Index. Therefore, the range for this indicator goes from +100 (all 3 answers were positive) to -100 (all three answers were negative).

Consumer Sentiment Index. The University of Michigan's Consumer Research Centre carries out a monthly survey among consumers, based on which a series of indicators are obtained.

The five questions included in their survey are:

1. (x₁) We are interested in people's current financial situation. Would you say that you (and the family you live with) are financially better or worse than a year ago?
2. (x₂) Looking ahead, do you think that a year from now you (and the family you live with) will be financially better off, worse off, or the same as now.
3. (x₃) Going back to the economic situation of the country as a whole, do you think that the next twelve months we will be a good financial period, a bad financial period, or otherwise?
4. (x₄) Looking ahead, would you say that it is more likely that, in the next 5 years, we will enjoy a sustained good period, or we will experience generalised unemployment, depression, or otherwise?
5. (x₅) As for those things that people buy for their homes -such as furniture, a fridge, a TV, that sort of thing- generally speaking, do you think that now is a good time or a bad time to buy large appliances or articles for the home?

To calculate the Consumer Sentiment Index (CSI) first of all the relative scores (the percentage of positive answers minus the percentage of negative answers, plus 100) are calculated for each of the 5 questions. Then the results are rounded up to the nearest integer. Using the following formula, the 5 relative scores are added together, they are divided by the relative score of the base year (1996), which is 6.7558, and then 2.0 (a constant the aims at correcting changes in the design of the 50s sample) is added:

Applying the same procedure, the Current Economic Conditions Index (ICC) and the Consumer Expectations Index (ICE) is calculated, using the following formulas:

Economic Optimism Index. Created by the TechnoMetrica Institute of Policy and Politics (TIPP) in the United States, this index is based on a monthly telephone survey carried out on approximately 1,000 adults. The questions to determine this indicator are:

1. In the next six months, do you think the national economic situation will be better, worse or similar to the present?
2. In the next six months, do you think your personal economic situation will be better, worse or similar to the present?
3. How do you feel about the current federal economic policies when it comes to keeping the economy going in a good direction: very satisfied, somewhat satisfied, not very satisfied or not at all satisfied?

It is calculated as follows:

• If the question has a positive answer and a negative answer (better and worse, approve and disapprove), the index component for this question is the following:

  

  where p₊ is the percentage of positive answers, and p_- the percentage of negative answers.

• If the question allows a choice between two positive questions and two negative questions (very satisfied, somewhat satisfied, not very satisfied or not at all satisfied), the index component then uses the following formula:

  

  where p₊₊ is the percentage of strongly positive answers, p_-- the percentage of strongly negative answers, p₊ the percentage of moderately positive answers, and p_- the percentage of moderately negative answers.

After this, the average of the question components is obtained to calculate the aggregate index. When the index level goes above 50, that means that the positive answers outnumber the negative answers. If all answers were strongly positive, the index would be 100, while if all answers were strongly negative, the index would be 0.

Presidential Leadership Index. Also created by the TIPP, this index is based on an analogous monthly telephone survey. The questions to determine this indicator are:

1. Overall, is your opinion of George W. Bush generally favorable, generally unfavorable, or are you not familiar enough to say one way or another?
2. In general, do you approve or disapprove of how George W. Bush is handling his job as president, or are you not familiar enough to say one way or the other?
3. How would you describe the leadership that president Bush is providing for the country? Would you say it is very strong, strong, moderate, weak or very weak?

This index is calculated in the same way as the previous one.

National Perspective Index. Also created by the TIPP, this index is also based on an analogous monthly telephone survey. The questions are:

1. In general, how satisfied are you with the direction things are taking in the country at the moment? Would you say you are very satisfied, somewhat satisfied, not very satisfied, or not at all satisfied?
2. Generally speaking, how satisfied are you with the direction things are taking at the moment in terms of morality and ethics? Would you say you are very satisfied, somewhat satisfied, not very satisfied, or not at all satisfied?
3. Generally speaking, in the next six months, do you think the position of the United States as a world leader will be better, worse, or the same as now?
4. In the next six months, do you think your quality of life will be better, worse, or about the same as now?

This index is also calculated in the same way as the ones above.

CALCULATING BAROMETER INDICATORS

As already indicated, direct answer percentages are sometimes, in themselves, indicators of the events they describe. A valid example would be the percentages of voting intention in respect of each political party. In these cases, it is not necessary to create additional indicators.

In this section we will describe the general calculation procedure for those questions for which it would be of interest to create an indicator. We will take the case of an isolated question, since, as we saw in the previous section, this is the point in which the different methods differ. When it comes to adding up the results of several questions, in all cases the arithmetic average of the indicators for each question is taken as indicator for the whole set.

Upon comparing the calculation methods shown in the previous section, the following can be observed:

• The indices calculated for the Consumer Comfort Index use values between -100 and 100. These are harder to interpret than those that use only positive values, and they are particularly complicated when one wants to interpret them as a whole, in relation to the same base period.
• In the cases in which there are three answer choices (positive, null, and negative) with answer ratios p₊, p₀, and p_- respectively, the answer balances are generally used, since they coincide with the variable's average:

  

• The base period of the Consumer Confidence Index is the year 1985, while the base period of the Consumer Sentiment Index is 1996. In index number theory, the purpose of taking a base period is to put the values in the same magnitude, so they may be compared and computed jointly. In other words, when we have measurements in a physical magnitude for each period (such as production volume in kg, litres, etc.), we divide it by the base period magnitude in order to obtain comparable values -the ratio in relation to the base period.

In this particular case -answers to questions that yield qualitative variables- the results are taken in the form of answer ratios for each category, values which are already directly comparable, so it does not seem necessary to take a base period (later on one can always create all the indices for a base period by just dividing them by the arithmetic average of that period).

On the other hand, selecting a base period allows a certain degree of manipulation which can affect the validity of these indices. For example, see on Graph 1 the series of scores between 0 and 100, Series1 and Series2 from the year 2000 to the year 2002. These scores are directly comparable and it can be seen that in 2002 they both have the same value of 50.

We can take the year 2000 as base, which gives the series that are shown on Graph 2.

However, we can also take the year 2001 as base, obtaining Graph 3.

That is to say, we can choose a base year that would allow us to see Series1 or Series2 more positively in the year 2000, as convenient, even when the scores for that year are identical. To avoid this flexibility, it seems more reasonable not to take a fixed base period for values that are directly comparable.

• The Consumer Confidence Index of the Conference Board makes a prior adjustment to account for seasonality in the series of ratios for each answer choice. However, the answer ratio of the CIS barometer series do not have a significant seasonal component, so it does not seem necessary to add this step, which would complicate the creation of indicators. On the other hand, when it comes to creating aggregate indicators, it is open to debate whether it is better to seasonally adjust and then add the indicators, or vice versa.

Taking what has been stated above into consideration, we try to create indicators without the mentioned disadvantages, while at the same time endeavouring to make them as simple as possible.

The questions for which we want to create indicators are the ones with multiple choice answers of the kind:

• Positive
• Null
• Negative

or of the kind:

• Very Good
• Good
• So-so
• Bad
• Very bad

In other words, multiple choice answers in which a scale can be observed. For this reason, a value or score can be associated to each answer choice. We choose scores between 0 and 100, from worst to best, since this type of score is easy to use. We allocate these scores from the Bayesian point of view, where there is no reason that keeps us from assuming equidistance. Therefore, in the event of 5 answer choices, we would have:

and, in the event of three options:

The indicator we choose is the arithmetic average of the scores weighted with the answer ratios. This way, in the case of 5 options, if p1, p2, p3, p4 and p5 are the answer ratios that correspond to the different options, the indicator will be:

The calculation would be analogous in the case of having three options, or in the case of not having a central category or option.

This indicator seems to be the most "natural" one for variables whose answer options run on a scale, and it has the advantage of being very simple.

In addition, its calculation matches that of other indicators of an apparently more sophisticated construction: the Economic Optimism Index, the Presidential Leadership Index and the National Perspective Index, created by the TIPP.

In fact, if we have 5 answer choices, the TIPP calculation would be:

An aspect to consider when creating CIS indicators is the fact that, in this case, it is not about designing a questionnaire with questions that are considered ideal for it, but rather of trying to take advantage of questions that have been used for years with a similar methodology. For this reason, sometimes the created indicators cannot be considered the best.

In the next sections, we will describe all the indices and indicators that are created from the barometers. They are grouped by theme, in the same way as they are created.

ECONOMIC SITUATION INDICATORS

In the barometers from every month (except August, when the survey is not conducted), there are two questions that deal with the economic situation:

• Q1: Referring now to the general economic situation in Spain, how would you rate it: very good, good, so-so, bad, or very bad?
• Q2: And, do you think that in a year from now the economic situation in Spain will be better, the same, or worse than now?

Sometimes, we also ask about situations in retrospect, but since these have not being made systematically since 1996, they are not considered when creating the indicators.

The Current Economic Situation indicator (SEA) is created based on question P1 as:

where p₁, p₂, p₃, p₄ and p₅ are the ratios for these answer options: very good, good, so-so, bad, and very bad, respectively.

Similarly, we create the Economic Expectations Index (IEE) based on question P2 as:

where p₁, p₂, p₃, p₄ and p₅ are the ratios for these answer options: better, same, and worse, respectively.

Lastly, the Economic Confidence Indicator (ICE) is the arithmetic average of the other two:

Graphs 4 to 7 show these indicators since 1996.

POLITICAL SITUATION INDICATORS

The monthly barometer questions regarding the political situation are:

• Q1: Referring now to the general political situation in Spain, how would you rate it: very good, good, so-so, bad, or very bad?
• Q2: And, do you think that in a year from now the political situation in Spain will be better, the same, or worse than now?

In other words, these questions are totally analogous to those of the economic situation. Sometimes, we also ask about situations in retrospect, but since these have not being made systematically since 1996, they are not considered when creating the indicators. These will be built similarly.

The Current Political Situation Indictor (SPA), based on question P1, will be:

where p₁, p₂, p₃, p₄ and p₅ are the ratios for these answer options: very good, good, so-so, bad, and very bad, respectively.

The Political Expectations Indicator (IEP), based on question P2, will be:

where p₁, p₂, p₃, p₄ and p₅ are the ratios for these answer options: better, same, and worse, respectively.

Lastly, the Political Confidence Indicator (ICF) is the arithmetic average of the other two:

Graphs 8 to 11 show these indicators.

POLITICAL SYSTEM INDICATORS

The barometer questions considered for these indicators are:

• Q1: As a whole, how would you rate the performance of the PSOE government: very good, good, so-so, bad, or very bad?
• Q2: As a whole, how would you rate the performance of the PP government: very good, good, so-so, bad, or very bad?
• Q3: In general, how would you rate the political performance of the PSOE as opposition party: very good, good, so-so, bad, or very bad?
• Q4: In general, how would you rate the political performance of the PP as opposition party: very good, good, so-so, bad, or very bad?

In the barometers for the first month of each quarter since 1996 we have been asking questions P1 and P4, or P2 and P3 alternately, depending on which is the party that is currently in power, and which is the main opposition party.

After these questions, first we create the "Government's Performance Evaluation" series, and the "First Opposition Party's Performance Evaluation" series, just by inserting questions P1 and P2 on one side, and P3 and P4 on the other, respectively (they evaluate the performance of the government and the opposition, regardless of the party to which each one belongs to).

Following this, we create the indicators:

• Government Performance Indicator (ILG):

  

  where p₁, p₂, p₃, p₄ and p₅ are the ratios for these answer options: very good, good, so-so, bad, and very bad, respectively, in questions P1 and P2.

• Opposition Performance Indicator (ILO):

  

  where p₁, p₂, p₃, p₄ and p₅ are the ratios for these answer options: very good, good, so-so, bad, and very bad, respectively, in questions P3 and P4.

We then build the Political System Confidence Indicator (ICSP) as the arithmetic average of the other two:

Graphs 12 to 15 show these indicators.

ELECTORAL INDICATORS

Electoral series are obtained from the following questions in the barometers of the first month of each quarter:

- If general elections for the Spanish Parliament were held tomorrow, what party would you vote for?

The first indicator is known as Voting Intention, and is merely a series of answer percentages for each political party. The series is an indicator in itself, and does not require further elaboration.

The result for the main political parties are shown in Graphs 16 and 17.

However, the answers to this question do not constitute a good prediction if the objective is to get as close as possible to the results of coming elections, since the percentages of answers are too high for the "Does not know"; and "No reply" options.

To better predict election results, the CIS creates what is known as Voting Prediction based Voting Intention, which is achieved by applying a correction model to the Voting Intention, based on other variables of the survey, past experiences, qualitative information, etc.

For this reason, Voting Prediction as an indicator does not compare to the others presented, since its calculation method has never been made public, and has changed along with the CIS management teams. It could be said that it does not constitute an authentic temporal series, but, given its importance, it is presented within the election indicators and shown in Graphs 18 and 19 for the main parties.

IDEOLOGICAL SELF-PLACEMENT

Based on the question:

- When talking about politics, we normally use the expressions "left wing" and "right wing". In this card there are a series of cells going from left (value 1) to right (value 10). In which cell would you place yourself?

The Ideological Self-placement indicator is created as the arithmetic average of the answers (eliminating the "Does not know" and "No reply" choices). Since this is a quantitative variable, the average is the simplest and best indicator to choose.

In addition, the typical deviation, which is almost constant, is usually calculated to verify the validity of the average calculation.

Graph 20 shows the indicator.

It is important to point out that this indicator leans to the left in a degree that is hard to evaluate. Using a scale from 1 to 10 produces this undesirable effect (probably used to avoid the negative connotations of assigning a zero value to the far left position).

In fact, upon studying the answer percentages for each score from 1 to 10, it can be observed that value 5 is systematically popular at all times. This seems to indicate that most participants interpret value 5 as the middle value, since they are used to dealing, in many situations, with 0 to 10 scales in which 5 is the middle value. However, in a scale from 1 to 10, such as the one used for this question, the middle value is 5.5, and therefore the arithmetic average obtained will lean more to the left than it should.

BIBLIOGRAPHY

• " Essays in Index Number Theory", W.E. Diewert, A.O. Nakamura. Ed. North-Holland (1993).
• " The TIPP Indices Methodology".
• " Consumer Confidence Survey Methodology".
• " The ABC News/Washington Post Consumer Comfort Index: Methodology".
• " Index calculations. Survey of Consumers. University of Michigan".
• " Nota Metodológica sobre los Indicadores del Barómetro". Pilar Rey del Castillo. Revista Española de Investigaciones Sociológicas nº108. Centro de Investigaciones Sociológicas. Octubre-Diciembre 2004.