Web scraping to build social research data (online) (join a waiting list)
Date:
17/05/2021
Organised by:
The University of Edinburgh
Presenter:
Dr Diarmuid McDonnell, Research Fellow, Third Sector Research Centre, University of Birmingham
Level:
Entry (no or almost no prior knowledge)
Contact:
Laura Marshall (Monday - Wednesday)
L.marshall@ed.ac.uk
Currently working from home following government guidance
Venue: Online
Description:
There is an unprecedented amount of information on the internet that could usefully be harvested in order to build social science research datasets.
This one-day online course will showcase suitable techniques for web scraping.
The value, logic and process of capturing data stored on websites will be described in detail, and practical examples and exercises will be demonstrated using the Python programming language.
It is most suited to empirical social science researchers.
Course Contents
- The structure of a web page and how it can be navigated to collect data
- Scraping, formatting and saving data scraped from the web
- Ethical and legal considerations when scraping data
- Introduction to the Python programming language
Learning Outcomes
Participants will gain
- A clearer understanding of the concept of web scraping
- An understanding of the fundamental skills required for scraping a web page
- An understanding of how to marshal scrapped data and construct a dataset suitable for social research
- An awareness of the scope and limitation of web scraping approaches
- An awareness of the theoretical, practical and ethical issues relating to scraping data
- Experience and examples of using the Python programming language for scraping data from a web page
The course will be delivered online and will be held from 10:00 - 16:00.
Course Programme
10:00 – 10:10 Welcome
10:10 – 10:45 The value, logic and practice of web scraping
10:45 – 11:00 Tea Break
11:00 – 12:00 Web scraping using Python: Example 1
12:00 – 13:00 Lunch
13:00 – 13:45 Web scraping using Python: Example 2
13:45 – 14:00 Tea Break
14:00 – 14:50 Practical exercise
14:50 – 15:00 Break
15:00 – 15:30 Ethical and legal considerations of web scraping
15:30 – 16:00 Concluding remarks
Cost:
The fee per teaching day is: • £30 per day for UK/EU registered students • £60 per day for staff at UK/EU academic institutions, UK/EU Research Councils researchers, UK/EU public sector staff and staff at UK/EU registered charity organisations and recognised UK/EU research institutions. • £100 per day for all other participants. Cancellation Policy: Please be reminded that you will be charged the full registration fee if you cancel your place within 4 weeks before the training delivery date or you fail to attend. If you are able to fill the place on the course you are cancelling then the cancellation charge will not apply
Website and registration:
Region:
Scotland
Keywords:
Digital Social Research, Big data, Web scraping, Python, Digital Social Research, research ethics, online data collection
Related publications and presentations from our eprints archive:
Digital Social Research
Big data