Class KuromojiTokenizer

Inheritance

object

TokenizerBase

KuromojiTokenizer

Implements

IKuromojiTokenizer

ITokenizer

Inherited Members

TokenizerBase.Type

TokenizerBase.Version

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.MemberwiseClone()

object.ReferenceEquals(object, object)

object.ToString()

Namespace: OpenSearch.Client

Assembly: OpenSearch.Client.dll

Syntax

public class KuromojiTokenizer : TokenizerBase, IKuromojiTokenizer, ITokenizer

Constructors

KuromojiTokenizer()

Declaration

public KuromojiTokenizer()

Properties

DiscardCompoundToken

Whether original compound tokens should be discarded from the output with SearchMode. Defaults to false.

Declaration

public bool? DiscardCompoundToken { get; set; }

Property Value

Type	Description
bool?

DiscardPunctuation

Whether punctuation should be discarded from the output. Defaults to true.

Declaration

public bool? DiscardPunctuation { get; set; }

Property Value

Type	Description
bool?

Mode

The tokenization mode determines how the tokenizer handles compound and unknown words.

Declaration

public KuromojiTokenizationMode? Mode { get; set; }

Property Value

Type	Description
KuromojiTokenizationMode?

NBestCost

The nbest_cost parameter specifies an additional Viterbi cost. The KuromojiTokenizer will include all tokens in Viterbi paths that are within the nbest_cost value of the best path.

Declaration

public int? NBestCost { get; set; }

Property Value

Type	Description
int?

NBestExamples

The nbest_examples can be used to find a nbest_cost value based on examples. For example, a value of /箱根山-箱根/成田空港-成田/ indicates that in the texts, 箱根山 (Mt. Hakone) and 成田空港 (Narita Airport) we’d like a cost that gives is us 箱根 (Hakone) and 成田 (Narita).

Declaration

public string NBestExamples { get; set; }

Property Value

Type	Description
string

UserDictionary

The Kuromoji tokenizer uses the MeCab-IPADIC dictionary by default. A user_dictionary may be appended to the default dictionary.

Declaration

public string UserDictionary { get; set; }

Property Value

Type	Description
string

UserDictionaryRules

Inline rule version of UserDictionary

Declaration

public IEnumerable<string> UserDictionaryRules { get; set; }

Property Value

Type	Description
IEnumerable<string>

Implements

IKuromojiTokenizer

ITokenizer

Extension Methods

SuffixExtensions.Suffix(object, string)