Digital Technology

M010: Natural Language Processing and Digital Philological Analysis for Asian Languages

Organizer(s)

• JT

  Jeffrey Tharsen

  The University of Chicago, United States

Chair(s)

• JT

  Jeffrey Tharsen

  The University of Chicago, United States

Presenter(s)

• JT

  Jeffrey Tharsen

  The University of Chicago, United States

In this workshop, participants will gain practical hands-on training in how to use natural language processing (NLP) approaches to perform digital philological analyses of source texts or even entire corpora in any Asian language.

The NLP/analytical methods covered in this session include:

• Optical charater recognition, cleaning and formatting source texts


• Tokenization, part-of-speech tagging and lemmatization


• Named entity recognition


• Topic modeling


• Stylometry


• Word2vec and Doc2vec


• TF-IDF


• Sentiment analysis


• Neural Network-based text modeling and text generation

Depending on interest and time, participants can also learn to perform digital phonological and/or intertextual analyses of their sources, and learn best practices for visualizing the results generated by various NLP frameworks.
Participants need no prior programming experience to participate, though a basic knowledge of Python would be beneficial for more advanced techniques. All participants should bring their own laptops, preferably with Java and Anaconda3 (for those wishing to use Python) preinstalled.
All participants should also bring at least a sample of the digitized source texts they wish to analyze: PDFs/scans or digital editions, in any Asian language, ancient to modern, and at any scale.
 
Learning objectives:

• Participants will learn how to prepare their sources for algorithmic data analytical frameworks and gain a brief overview of modern OCR best practices.


• Participants will learn how to apply a full spectrum of current natural language processing tools to any corpus in any Asian language.


• Participants will learn to interpret the results from various NLP analyses, evaluate the algorithms and tools, and gain a basic understanding of their underlying methods.

Relevant websites and bibliography:
Tesseract 4: https://github.com/tesseract-ocr/tesseract
Stanford CoreNLP: https://stanfordnlp.github.io/CoreNLP

TopicModelingTool: https://github.com/senderle/topic-modeling-tool

Lexos: http://lexos.wheatoncollege.edu

SpaCy and Gensim (Python): https://spacy.io , https://radimrehurek.com/gensim

Multilingual BERT: https://github.com/google-research/bert/blob/master/multilingual.md

GPT-2: https://github.com/openai/gpt-2 , https://github.com/imcaspar/gpt2-ml


Edward Slingerland, Ryan Nichols, Kristoffer Neilbo, Carson Logan, “The Distant Reading of Religious Texts: A ‘Big Data’ Approach to Mind-Body Concepts in Early China,” Journal of the American Academy of Religion, Volume 85, Issue 4, December 2017. https://doi.org/10.1093/jaarel/lfw090

Paul Vierthaler and Mees Gelein, “A BLAST-based, Language-agnostic Text Reuse Algorithm with a MARKUS Implementation and Sequence Alignment Optimized for Large Chinese Corpora,” Journal of Cultural Analytics. March 18, 2019. https://doi.org/10.31235/osf.io/7xpqe