Machine Learning Assisted Text Annotation

Combining human annotation and your machine predictions to deliver high quality annotated text for your NLP Projects

Here’s how it works

First, you upload plain text to annotate or a JSON of custom tokens, spans, and predictions. Quickly configure classes for annotators with custom class instructions and in-tool help functionality. Create flexible test questions to evaluate annotators on comprehension and accuracy. Finally, launch jobs and aggregate correct labels from multiple annotators with an inter-annotator agreement score. See where your model required human correction.

Label Experience for Machine Learning Assisted Text Annotation

Human labelers quickly assign class labels to individual tokens and spans for entity extraction and parts of speech labeling.  Machine Learning Assisted Text Annotation can be used for chatbots, search science, document understanding and other use cases.  Leverage your own tokenizers or use spaCy, NLTK or Stanford in Figure Eight’s platform to create tokens quickly in English and 18 other languages.  Bring your own model predictions to help annotators increase precision, recall, and speed.

An easy and efficient labeling experience

  • Hotkey support for major functionality allowing annotators to optimize their process
  • In-tool customizable lookup for tokens and spans aids reading comprehension
  • Label all instances of tokens or spans simultaneously and minimize clicks
  • See annotation context and instructions in-line to quickly see guidance and tips
  • No highlighting required!

High-Quality Low Bias Labels

  • Test Questions test for precise class labels and span lengths
  • Inter-annotator agreement scores combine the best of multiple annotators
  • Pass through machine predictions to increase speed and accuracy and get more precise data for retraining

Simple model validation and retraining

Upload your predictions for human annotator review. Every annotation is recorded as being “Machine” or “Human” labeled to track class level accuracy, recall, and precision. Reduce annotator bias with test questions that can test for true negatives and false positives from the model or the annotator. Feed low-performing corrected labels back into model for retraining to reduce error rate quickly and cost-effectively.

Interested in trying out our Machine Learning Assisted Text Annotation solution?
Sign up to talk with one of our experts and we’ll help you get set up.

Figure Eight is trusted by today’s leading brands