textfier.stream

Every pipeline has its first step, right? The stream package serves as primary methods to clean and tokenize data.

• textfier.stream.cleaner
  • clean_sentences
  • clean_words
• textfier.stream.tokenizer
  • tokenize_to_sentences
  • tokenize_to_words

Stream package that provides cleaning and tokenization utilities.