Research Design and Methods Introduction to Quantitative Research and SPSS

Preoccupations of quantitative researchers

•       Measurement – external facts can be measured (without influence from the researcher) 

•       Causality – why are things the way they are? What is the association between variables?

•       Generalisation – being able to say the findings would apply in a wider population

•       Replication – if the study was repeated, it would produce the same results

What is a survey?

•       A survey is not just a particular technique of collecting information: questionnaires are widely used but other techniques, such as structured and in-depth interviews, observation, content analysis and so forth, can also be used in survey research.  The distinguishing feature of surveys are the form of data and method of analysis” (De Vaus, 2002, p. 3)

•       “Surveys are characterised by a structured or systematic set of data… all this means is that we collect information about the same variables or characteristics from at least two (normally far more) cases… Since the same information is collected for each case the cases are directly comparable and we end up with a structured set of data” (De Vaus, 2002, p.3)

The components of a quantitative survey

1. The research instrument – usually the questionnaire -pretested
2. A method of implementation – online, face to face, postal, telephone.  Piloted.
3. Methods of administration – inc. recording and monitoring of  responses, cover letters, advance notification
4. A sampling strategy – including determination of sample size and method
5. A strategy for dealing with non-response
6. A system for processing, editing and cleaning data (SPSS)
7. Analysis of sample composition & representativeness
8. A quantitative statistical analysis of the data matrix – inc. analysis of variation and appropriate statistical output
9. Presentation of analysis – including tabular, graphical and statistical presentations
10. The report – including an evaluation of the strengths and weaknesses of the survey and any caveats associated with data interpretation.

Planning survey research

•       Identifying the population and the sampling framework

•       Design of the survey (what questions to ask and how)

•       Considering the analysis you will need to undertake to answer you research questions

•       Identifying the statistical tests to use and why

•       Much of this is done BEFORE data collection/analysis, i.e. careful design a priori

Important notions for research design:
samples and population

•       POPULATION:

                The entire set of cases under discussion

                (e.g. all visitors who have visited Lincoln cathedral)

•       SAMPLE:

                The set of cases that you are collecting data on

                (e.g. the visitors to the cathedral who complete the cathedral survey)

‘N’ is the number of these responses

Why sample?

Making models of the ‘whole population’ from samples drawn from it.

                Each sample will have a mean value that will vary from the ‘true mean’ for the population (which we may never really know) but we can expect the value of a sample to be within a certain range of the true mean (if the sample is randomly drawn from the same population).

                We can use this to give us some measure of the confidence of our estimates.

Types of sample/sampling method

probability sampling

•       Simple random sample

•       Stratified random sampling

non-probability sampling

•       Target sampling

•       Convenience sampling

•       Snowball sampling

How Big a Sample? When to stop?

•       For small samples you would need to use other statistical tests/distributions. In general, bigger is better.

•       General rule of thumb is usually ‘bigger is better’

•       The Economist’s (1997) rule of thumb is minimum of 30 items of data in a statistical analysis

How representative are the results?

•       Depends on: (1) number of responses (the more the better) and (2) heterogeneity of responses

               

Sample sizes required for sampling errors at 95% confidence level (for random samples)

Sample Size	Sampling Error
10,000 1,100 400 204 100	1% 3% 5% 7% 10%

Important notions for survey design:
independent and dependent variables

•       INDEPENDENT VARIABLES’:

                The variables in your survey that are INDEPENDENT of this particular survey  (e.g. the questions relating to a respondent’s personal information such as their gender)

•       DEPENDENT VARIABLES’:

                The variables that are not independent … (e.g. the variables that relate to questions about a particular service or product)

About SPSS

•       Statistical Package for the Social Sciences

•       Used in: market research, academia, government, health sector, education, human resources

•       Designed for analysis of large datasets, such as survey results, company records

•       Functions include: frequencies, cross-tabs, tables, graphs, correlations, among others as well as ability to calculate confidence levels etc

Data Presentation in SPSS

Data stored in SPSS is presented as a matrix. The variable names head the columns, and the information for each case or record is displayed across the rows.

 Types of Quantitative Data

Nominal	Data is not numerical, but is defined by different characteristics (male and female; student, employed, unemployed)
Ordinal	Nominal data that can be ranked (rating scales; age groups – 18-24, 25-34, 35-44)
Interval or Continuous	Values are measured numerically (e.g. age in years, salary, turnover, sales, distance)

Presenting Quantitative Data

	Presented Using	Measuring “Typical Response”
Nominal	Frequency Table Bar Chart (space between the bars) Pie Chart	Mode – the most common response
Ordinal	Frequency Table Bar Chart (space between the bars) Pie Chart	Median – the middle response when values are ranked
Interval or Continuous	Histogram (no space between the bars)	Mean – the average response

Missing Values

•       Missing values are non-responses - where questions haven’t been completed, or are not relevant to the respondent.

•       In most cases, responses should be presented without missing values – in other words, the VALID response.  Take care to provide details of number of valid/invalid responses.

•       Consider “don’t know” and “no opinion” responses – do you want to include these in your analysis?  They may provide insight into areas that respondents aren’t familiar with, or are undecided about.

Labelling

All tables and charts should include:

•       Number and Title (e.g. Graph 1: Age Breakdown of Sample)

•       Labels for variables on rows and columns (or axis if a chart), including whether %, whole numbers, currency

•       If presenting percentages, provide details of number of responses too (e.g. at bottom of graph “based on 60 valid responses”)

               

                Remember that presenting data in tables and charts does not constitute analysis.  Write a clear narrative around your data to explain the figures, and to outline what the data is telling you

Univariate and Bivariate Analysis

Univariate: Analysis of ONE variable, e.g. gender breakdown in your sample.  Can be presented using tables, bar charts, histograms, pie charts 

 Bivariate: Analysis of TWO variables, e.g. relationship between gender and salary.  Can also use cross-tabulations, line graphs, scatter graphs

Univariate Analysis

•       Frequencies

	Number	%
Agree	40	50%
Don’t Know	30	37.5%
Disagree	10	12.5%
TOTAL	80	100%

•       Descriptives

	N	Min	Max	Mean	Std. Deviation
How many people does your business normally employ? - Full Time	71	0	1500	32.15	180.513
Valid N	71

The ‘composition’ of your sample (sample composition):

What is this?

•       Is what your sample is composed of i.e. who is in your sample (what are the personal characteristics of the respondents in your survey?)

•       Is a summary of the independent variables in your database

Should be summarised :

Sample Composition	%
Males Females	55% 45%
Under 25s 25-35 years Over 35s	20% 70% 10%
UK EU Other	20% 40% 40%
N = 72

Looking at differences by independent variable: cross-tabs

   ‘Cross-Tabs’

(for nominal and

ordinal variables)

               

•       Need to decide on the independent (cause) and dependent (effect) variable to read percentages in a cross-tab. 

•       “If independent variable is across the top, use column percentage and compare these across the table.  If the independent variable is on the side use row percentages and compare these down the table” De Vaus (1996) p. 159

	Male	Female
Like Chocolate	25 50%	40 80%
Don’t Like Chocolate	25 50%	10 20%
TOTAL	50 100%	50 100%

_{Why do bivariate analysis?}

•       Usually, to move beyond ‘description’ to ‘explanation’. To move beyond ‘how things are’ to ‘why things are the way they are’

•       Explanation can be theoretically informed (testing theory) or inductive (producing theory)

•       Measured effects are explained by measured causes.  In other words ‘explanation’ requires the analysis of relationships within the data.  Often relationships are explored between ‘independent’ and ‘dependent’ variables.  

•       Quantitative analysis is more than explanation – it also about prediction.   It involves inferring what is likely within a wider population, given certain conditions.

•       Design limitations imply that ‘explanations’ are probabilistic.  Hence, ‘attending class increases the likelihood of doing well in assessments’.