Do you ever feel that the data you need for your research is accessible but it’s not in a convenient table, such as company reports or building plans? Perhaps the information you need is spread out across many different documents? If only we could read and extract structured data from thousands of written documents.
In this course, we explore how to accomplish this task by combining web scraping, Optical Character Recognition (OCR), and Natural Language Processing (NLP). Over four weeks, we provide online lessons and interactive sessions to learn the fundamentals of these key technologies.
Topics
- Methods for extracting text and files from websites using tools such as rvest and how to avoid common pitfalls.
- Methods for extracting text from images, such as scans of written documents.
- Exploring technologies that can help automate data extraction from harvested text and a critical review of common data quality issues.
Format
This is an online course.
- Week 1: Watch pre-prepared video lectures about relevant theory and demonstration of example exercises. The topic is web scraping and OCR (~60 min). Interactive Online Session (~60 min).
- Week 2: Applying last week’s lessons to the example coding exercise or your own project (~60 min). Interactive Online Session (~60 min).
- Week 3: Watch pre-prepared video lectures about relevant theory and demonstration of example exercises. The topic is NLP and common data extract issues (~60 min). Interactive Online Session (~60 min).
- Week 4: Interactive session about applying last week’s lessons to the example coding exercise or your own project (~60 min). Interactive Online Session (~60 min).
Weekly Meetings
The course includes 4 live Online Meetings, in which you will discuss the week’s contents with the instructor and fellow participants:
Meeting 1: Aug 27, 2024, 4:30pm – 5:30pm CEST
Meeting 2: Sep 03, 2024, 4:30pm – 5:30pm CEST
Meeting 3: Sep 10, 2024, 4:30pm – 5:30pm CEST
Meeting 4: Sep 17, 2024, 4:30pm – 5:30pm CEST
Prerequisites
- Basic programming knowledge (R, python, …)
- Willingness to learn new technical skills
About the Instructor
John ‘Jack’ Collins is a PhD Student in Sociology at the Graduate School of Economic and Social Sciences. He holds a Bachelor’s of Sociology with Honours from the Australian National University. Jack has a Master’s degree in Data Science from James Cook University. His Master’s project was regarding predictive modelling for student attrition from sub-tertiary courses in Australia. During his Master’s studies, he also assisted in research projects regarding social attitudes and voting behaviour in Australia. Before starting PhD, Jack was a Senior IT Consultant specialising in data engineering, analytics and software development. Jack is interested in applying Data Science and IT to sociological research, particularly with regard to machine learning, analytics, and web applications.