Corpus Bootstrapping with NLTK

When it comes to natural language processing, general APIs and generic models are often far less accurate than you want. Or maybe the APIs you need don’t even exist. Either way, you can use “corpus bootstrapping” to create custom models and APIs. Corpus bootstrapping is a method of rapidly producing a custom corpus for training highly accurate natural language processing models. For example, suppose you want to do sentiment analysis for Spanish text, but you can only find APIs and models for English. Or you want to do phrase extraction for phrases that are not exactly noun phrases. Maybe you want to classify text but there’s no corpus in existence with the categories you’re interested in. All of these problems can be solved by iterating your way to a custom corpus for training custom models.

This talk will cover:

• creating a classified corpus from scratch
• generating a sentiment analysis corpus in Spanish by starting with an English corpus
• using simplified part-of-speech tags to quickly produce a custom corpus for phrase extraction
• training custom models with NLTK-Trainer

Code examples will be in Python using NLTK.