- Welcome
- Important notice
- About the author
- About this site
- Site guide + Search box
- Dedications
- Acknowledgments
- My personal pantheon (of the great and the good in survey research)
- Recent and planned activities
- Textbooks for Research Methods and Data Analysis
- 1: Survey Analysis Workshop (SPSS)
- 1a: Statistical concepts and methods
- 1b: Teaching with Survey Data
- 1c: Developing research projects using survey data
- 1d: Workshop and presentations for ASSESS (SPSS users in Europe)
- 2: Survey Research Practice
- 2a: Survey Research Methodology, Practice and Training
- 2b: Major survey series
- 3: Subjective Social Indicators (Quality of Life)
- 4: Survey Unit, Social Science Research Council (UK)
- 5a: Polytechnic of North London (1976-1992)
- 5b: Survey Research Unit (1978-1992)
- Village life in Normandy
- Contact
- Origins of the British Crime Survey
- British Crime Survey
British Social Attitudes 1983 onwards
Cumulative SPSS file 1983 - 2017
[New page 22 June 2016: last updated 18 March 2019]
Cumulative files 1983 onwards
Why did I let myself in for this?
As a survey researcher who has used SPSS for research and teaching since the early 1970s, I have always viewed SPSS as tool rather than an end in itself. In my hands-on teaching at both undergraduate and postgraduate level, I have always explained SPSS files from the point of view of the student in the classroom and have always tried to make them easy to understand and use, even for users with little or no experience of surveys, computing, SPSS or (in the 1970s) even typing. Having been considered quite successful at this by my students, I have used my experience to write not only the tutorials for my self-teaching course Survey Analysis Workshop (SPSS) but also the introductions to, and commentaries on, SPSS files for major surveys distributed by UKDS and other archives. See for example : British Social Attitudes: Exploring the SPSS files, Commentary on Understanding Society 2010 and the relevant sections of ONS National Well-being and MacInnes (2017) which are easy to follow even for absolute beginners and also for users without immediate access to SPSS.
Attempting analyses across years became increasingly frustrating as I encountered a range of anomalies, incompatibilities and inconsistencies, not to mention universally incomplete and/or incorrect specifications of measurement levels, missing values and value labels. Accordingly I set myself the task of generating a complete cumulative SPSS file containing the data from all waves from 1983 onwards (one colleague described this undertaking as Herculean: he was right!) to provide what will hopefully be a valuable resource for teachers, students and researchers.
Index to UKDS downloads for British Social Attitudes 1983 - 2014 is an Excel file detailing, for each wave 1983 - 2014, year of survey, link to UKDS, download filename, size of file, number of cases, number of variables, number of variables with non-numeric formats and the new working filename assigned to amended files. The amended *.sav files were sent to Natcen for approval and possible deposit with UKDS, but are now superseded.
Non-numeric variables in British Social Attitudes is a step-by-step account of identifying, in each wave, variables with the same name, but different formats. Several of these variables are specified as Strings with widths varying from A4 to A60, but some are in fact numbers. Others are dates or times in DATE or TIME format and one is in COMMA1. These and other factors prevent merging data from different waves using the SPSS command ADD FILES. It’s been quite complex and tedious tracking them all down, but I eventually managed to create cumulative files for 1983 – 1994 and 2011 – 2014. Merging 1995 - 2005 and 2006 - 2009 was more daunting, as several problems remained to be resolved, but I eventually managed to generate a draft cumulative file for the whole series. Much more meticulous and painstaking detective work and editing was required before a beta version was ready for public release.
Cumulative SPSS "mother" file
Acknowledgments
Custom-written Python code, freely and generously supplied by Jon Peck (retired Senior Software Engineer, IBM-SPSS) has saved me weeks if not months of painstaking needle-in-haystack searches. I also wish to thank Dr Chris Stride (Sheffield) who suggested using the sort facility in Excel to separate variable names with single (positive) missing values from those with paired (positive and equivalent negative) missing values.
Cumulative SPSS "mother" file 1983 - 2017
The cumulative SPSS "mother" file for years 1983 to 2014 was completed on 20 June 2016. The 2015, 2016 and 2017 waves have now been added. The file currently contains 102,900 cases and 11,590 variables. The data are under safeguarded access, but discussions are under way with Natcen and UKDS as to the best way to provide access, but a neat solution has been found which meets all requirements (see page British Social Attitudes: Making files from different years compatible)
For sure, some mini-glitches may remain, but to find and resolve these would at this stage be completely uneconomic of my time. However users are warned that, because metadata for repeated variables are taken from the most recent wave, the value labels for categories of some variables differ from those of earlier waves. This is particularly true of ordinal variables for income groups.
My initial accounts of the work (still in progress) and problems encountered to combine data from the separate waves into a single file are listed below. Although written in plain English, they contain a lot of technical information which may not be of immediate interest to those wishing to pursue or teach substantive research topics using data from the BSA survey series. However they are important for those wishing to employ best practice in generating SPSS files for use by themselves and others. They describe exploration of the metadata step-by-step to find anomalies, inconsistencies and incompatibilities, and show how they were identified and corrected. Further commentaries on the later stages will be added as and when they are completed.
1983 to 2014
1: Constructing cumulative files
2: Resolution of type conflicts
3: Beta version of cumulative file
4.1_Preliminary identification of measurement levels and missing values
4.2_Further identification of measurement levels and missing values
4.3_Search for (missing) values 8 and 9 at beginning of value labels
4.4_Sequence of operations
2009 to 2014 only
British Social Attitudes 2009 - 2014: Measurement levels
British Social Attitudes 2009 to 2014: Missing values
British Social Attitudes 2009 to 2014: Multiple response
British Social Attitudes 2009 to 2014: Variable and value labels
Cumulative files 1983 onwards
Why did I let myself in for this?
As a survey researcher who has used SPSS for research and teaching since the early 1970s, I have always viewed SPSS as tool rather than an end in itself. In my hands-on teaching at both undergraduate and postgraduate level, I have always explained SPSS files from the point of view of the student in the classroom and have always tried to make them easy to understand and use, even for users with little or no experience of surveys, computing, SPSS or (in the 1970s) even typing. Having been considered quite successful at this by my students, I have used my experience to write not only the tutorials for my self-teaching course Survey Analysis Workshop (SPSS) but also the introductions to, and commentaries on, SPSS files for major surveys distributed by UKDS and other archives. See for example : British Social Attitudes: Exploring the SPSS files, Commentary on Understanding Society 2010 and the relevant sections of ONS National Well-being and MacInnes (2017) which are easy to follow even for absolute beginners and also for users without immediate access to SPSS.
Attempting analyses across years became increasingly frustrating as I encountered a range of anomalies, incompatibilities and inconsistencies, not to mention universally incomplete and/or incorrect specifications of measurement levels, missing values and value labels. Accordingly I set myself the task of generating a complete cumulative SPSS file containing the data from all waves from 1983 onwards (one colleague described this undertaking as Herculean: he was right!) to provide what will hopefully be a valuable resource for teachers, students and researchers.
Index to UKDS downloads for British Social Attitudes 1983 - 2014 is an Excel file detailing, for each wave 1983 - 2014, year of survey, link to UKDS, download filename, size of file, number of cases, number of variables, number of variables with non-numeric formats and the new working filename assigned to amended files. The amended *.sav files were sent to Natcen for approval and possible deposit with UKDS, but are now superseded.
Non-numeric variables in British Social Attitudes is a step-by-step account of identifying, in each wave, variables with the same name, but different formats. Several of these variables are specified as Strings with widths varying from A4 to A60, but some are in fact numbers. Others are dates or times in DATE or TIME format and one is in COMMA1. These and other factors prevent merging data from different waves using the SPSS command ADD FILES. It’s been quite complex and tedious tracking them all down, but I eventually managed to create cumulative files for 1983 – 1994 and 2011 – 2014. Merging 1995 - 2005 and 2006 - 2009 was more daunting, as several problems remained to be resolved, but I eventually managed to generate a draft cumulative file for the whole series. Much more meticulous and painstaking detective work and editing was required before a beta version was ready for public release.
Cumulative SPSS "mother" file
Acknowledgments
Custom-written Python code, freely and generously supplied by Jon Peck (retired Senior Software Engineer, IBM-SPSS) has saved me weeks if not months of painstaking needle-in-haystack searches. I also wish to thank Dr Chris Stride (Sheffield) who suggested using the sort facility in Excel to separate variable names with single (positive) missing values from those with paired (positive and equivalent negative) missing values.
Cumulative SPSS "mother" file 1983 - 2017
The cumulative SPSS "mother" file for years 1983 to 2014 was completed on 20 June 2016. The 2015, 2016 and 2017 waves have now been added. The file currently contains 102,900 cases and 11,590 variables. The data are under safeguarded access, but discussions are under way with Natcen and UKDS as to the best way to provide access, but a neat solution has been found which meets all requirements (see page British Social Attitudes: Making files from different years compatible)
For sure, some mini-glitches may remain, but to find and resolve these would at this stage be completely uneconomic of my time. However users are warned that, because metadata for repeated variables are taken from the most recent wave, the value labels for categories of some variables differ from those of earlier waves. This is particularly true of ordinal variables for income groups.
My initial accounts of the work (still in progress) and problems encountered to combine data from the separate waves into a single file are listed below. Although written in plain English, they contain a lot of technical information which may not be of immediate interest to those wishing to pursue or teach substantive research topics using data from the BSA survey series. However they are important for those wishing to employ best practice in generating SPSS files for use by themselves and others. They describe exploration of the metadata step-by-step to find anomalies, inconsistencies and incompatibilities, and show how they were identified and corrected. Further commentaries on the later stages will be added as and when they are completed.
1983 to 2014
1: Constructing cumulative files
2: Resolution of type conflicts
3: Beta version of cumulative file
4.1_Preliminary identification of measurement levels and missing values
4.2_Further identification of measurement levels and missing values
4.3_Search for (missing) values 8 and 9 at beginning of value labels
4.4_Sequence of operations
2009 to 2014 only
British Social Attitudes 2009 - 2014: Measurement levels
British Social Attitudes 2009 to 2014: Missing values
British Social Attitudes 2009 to 2014: Multiple response
British Social Attitudes 2009 to 2014: Variable and value labels