Skip to contents

Tokenize and Analyze Japanese Texts

Source: R/fledgingr.R
tokenize.Rd

Tokenize and Analyze Japanese Texts

Usage

tokenize(x, mode = c("C", "B", "A"))

Arguments

x

Text to tokenize.

mode

Unit to split text.

  • "A": short

  • "B": middle

  • "C": long