Interface Tokenizer

All Known Subinterfaces:
CompoundWordTokenizer, SentenceTokenizer
All Known Implementing Classes:
SimpleSentenceTokenizer, SRXSentenceTokenizer, WordTokenizer

public interface Tokenizer
Interface for classes that tokenize text into smaller units.
  • Method Summary

    Modifier and Type
    Method
    Description