Class TokenFiltersDescriptor
Inheritance
TokenFiltersDescriptor
Assembly: OpenSearch.Client.dll
Syntax
public class TokenFiltersDescriptor : IsADictionaryDescriptorBase<TokenFiltersDescriptor, ITokenFilters, string, ITokenFilter>, IDescriptor, IPromise<ITokenFilters>
Constructors
|
Edit this page
View Source
TokenFiltersDescriptor()
Declaration
public TokenFiltersDescriptor()
Methods
|
Edit this page
View Source
AsciiFolding(string, Func<AsciiFoldingTokenFilterDescriptor, IAsciiFoldingTokenFilter>)
A token filter of type asciifolding that converts alphabetic, numeric, and symbolic Unicode characters which are
not in the first 127 ASCII characters (the “Basic Latin” Unicode block) into their ASCII equivalents, if one
exists.
Declaration
public TokenFiltersDescriptor AsciiFolding(string name, Func<AsciiFoldingTokenFilterDescriptor, IAsciiFoldingTokenFilter> selector)
Parameters
Returns
|
Edit this page
View Source
CommonGrams(string, Func<CommonGramsTokenFilterDescriptor, ICommonGramsTokenFilter>)
Token filter that generates bigrams for frequently occuring terms. Single terms are still indexed.
Note, common_words or common_words_path field is required.
Declaration
public TokenFiltersDescriptor CommonGrams(string name, Func<CommonGramsTokenFilterDescriptor, ICommonGramsTokenFilter> selector)
Parameters
Returns
|
Edit this page
View Source
Condition(string, Func<ConditionTokenFilterDescriptor, IConditionTokenFilter>)
The conditional token filter takes a predicate script and a list of subfilters, and
only applies the subfilters to the current token if it matches the predicate.
Declaration
public TokenFiltersDescriptor Condition(string name, Func<ConditionTokenFilterDescriptor, IConditionTokenFilter> selector)
Parameters
Returns
|
Edit this page
View Source
DelimitedPayload(string, Func<DelimitedPayloadTokenFilterDescriptor, IDelimitedPayloadTokenFilter>)
Splits tokens into tokens and payload whenever a delimiter character is found.
Declaration
public TokenFiltersDescriptor DelimitedPayload(string name, Func<DelimitedPayloadTokenFilterDescriptor, IDelimitedPayloadTokenFilter> selector)
Parameters
Returns
|
Edit this page
View Source
DictionaryDecompounder(string, Func<DictionaryDecompounderTokenFilterDescriptor, IDictionaryDecompounderTokenFilter>)
Token filters that allow to decompose compound words using a dictionary
Declaration
public TokenFiltersDescriptor DictionaryDecompounder(string name, Func<DictionaryDecompounderTokenFilterDescriptor, IDictionaryDecompounderTokenFilter> selector)
Parameters
Returns
|
Edit this page
View Source
EdgeNGram(string, Func<EdgeNGramTokenFilterDescriptor, IEdgeNGramTokenFilter>)
A token filter of type edgeNGram.
Declaration
public TokenFiltersDescriptor EdgeNGram(string name, Func<EdgeNGramTokenFilterDescriptor, IEdgeNGramTokenFilter> selector)
Parameters
Returns
|
Edit this page
View Source
Elision(string, Func<ElisionTokenFilterDescriptor, IElisionTokenFilter>)
A token filter which removes elisions. For example, “l’avion” (the plane) will tokenized as “avion” (plane).
Declaration
public TokenFiltersDescriptor Elision(string name, Func<ElisionTokenFilterDescriptor, IElisionTokenFilter> selector)
Parameters
Returns
|
Edit this page
View Source
Fingerprint(string, Func<FingerprintTokenFilterDescriptor, IFingerprintTokenFilter>)
A token filter of type fingerprint The fingerprint token filter that emits a single token which is useful
for fingerprinting a body of text, and/or providing a token that can be clustered on.
It does this by sorting the tokens, deduplicating and then concatenating them back into a single token.
Declaration
public TokenFiltersDescriptor Fingerprint(string name, Func<FingerprintTokenFilterDescriptor, IFingerprintTokenFilter> selector = null)
Parameters
Returns
|
Edit this page
View Source
Hunspell(string, Func<HunspellTokenFilterDescriptor, IHunspellTokenFilter>)
Basic support for hunspell stemming.
Hunspell dictionaries will be picked up from a dedicated hunspell directory on the filesystem.
Declaration
public TokenFiltersDescriptor Hunspell(string name, Func<HunspellTokenFilterDescriptor, IHunspellTokenFilter> selector)
Parameters
Returns
|
Edit this page
View Source
HyphenationDecompounder(string, Func<HyphenationDecompounderTokenFilterDescriptor, IHyphenationDecompounderTokenFilter>)
Token filters that allow to decompose compound words using FOP XML
Declaration
public TokenFiltersDescriptor HyphenationDecompounder(string name, Func<HyphenationDecompounderTokenFilterDescriptor, IHyphenationDecompounderTokenFilter> selector)
Parameters
Returns
|
Edit this page
View Source
IcuCollation(string, Func<IcuCollationTokenFilterDescriptor, IIcuCollationTokenFilter>)
Collations are used for sorting documents in a language-specific word order. The icu_collation token filter is
available to all indices and
defaults to using the DUCET collation, which is a best-effort attempt at language-neutral sorting.
Part of the analysis-icu plugin:
Declaration
public TokenFiltersDescriptor IcuCollation(string name, Func<IcuCollationTokenFilterDescriptor, IIcuCollationTokenFilter> selector)
Parameters
Returns
|
Edit this page
View Source
IcuFolding(string, Func<IcuFoldingTokenFilterDescriptor, IIcuFoldingTokenFilter>)
Case folding of Unicode characters based on UTR#30, like the ASCII-folding token filter on steroids.
Part of the analysis-icu plugin:
Declaration
public TokenFiltersDescriptor IcuFolding(string name, Func<IcuFoldingTokenFilterDescriptor, IIcuFoldingTokenFilter> selector)
Parameters
Returns
|
Edit this page
View Source
IcuNormalization(string, Func<IcuNormalizationTokenFilterDescriptor, IIcuNormalizationTokenFilter>)
Declaration
public TokenFiltersDescriptor IcuNormalization(string name, Func<IcuNormalizationTokenFilterDescriptor, IIcuNormalizationTokenFilter> selector)
Parameters
Returns
|
Edit this page
View Source
Transforms are used to process Unicode text in many different ways, such as case mapping,
normalization, transliteration and bidirectional text handling.
Part of the analysis-icu plugin:
Declaration
public TokenFiltersDescriptor IcuTransform(string name, Func<IcuTransformTokenFilterDescriptor, IIcuTransformTokenFilter> selector)
Parameters
Returns
|
Edit this page
View Source
KStem(string, Func<KStemTokenFilterDescriptor, IKStemTokenFilter>)
The kstem token filter is a high performance filter for english.
All terms must already be lowercased (use lowercase filter) for this filter to work correctly.
Declaration
public TokenFiltersDescriptor KStem(string name, Func<KStemTokenFilterDescriptor, IKStemTokenFilter> selector = null)
Parameters
Returns
|
Edit this page
View Source
KeepTypes(string, Func<KeepTypesTokenFilterDescriptor, IKeepTypesTokenFilter>)
A token filter of type keep that only keeps tokens with text contained in a predefined set of words.
Declaration
public TokenFiltersDescriptor KeepTypes(string name, Func<KeepTypesTokenFilterDescriptor, IKeepTypesTokenFilter> selector)
Parameters
Returns
|
Edit this page
View Source
KeepWords(string, Func<KeepWordsTokenFilterDescriptor, IKeepWordsTokenFilter>)
A token filter of type keep that only keeps tokens with text contained in a predefined set of words.
Declaration
public TokenFiltersDescriptor KeepWords(string name, Func<KeepWordsTokenFilterDescriptor, IKeepWordsTokenFilter> selector)
Parameters
Returns
|
Edit this page
View Source
KeywordMarker(string, Func<KeywordMarkerTokenFilterDescriptor, IKeywordMarkerTokenFilter>)
Protects words from being modified by stemmers. Must be placed before any stemming filters.
Declaration
public TokenFiltersDescriptor KeywordMarker(string name, Func<KeywordMarkerTokenFilterDescriptor, IKeywordMarkerTokenFilter> selector)
Parameters
Returns
|
Edit this page
View Source
KuromojiPartOfSpeech(string, Func<KuromojiPartOfSpeechTokenFilterDescriptor, IKuromojiPartOfSpeechTokenFilter>)
The kuromoji_part_of_speech token filter removes tokens that match a set of part-of-speech tags.
Part of the analysis-kuromoji plugin:
Declaration
public TokenFiltersDescriptor KuromojiPartOfSpeech(string name, Func<KuromojiPartOfSpeechTokenFilterDescriptor, IKuromojiPartOfSpeechTokenFilter> selector)
Parameters
Returns
|
Edit this page
View Source
The kuromoji_readingform token filter replaces the token with its reading form in either katakana or romaji.
Part of the analysis-kuromoji plugin:
Declaration
public TokenFiltersDescriptor KuromojiReadingForm(string name, Func<KuromojiReadingFormTokenFilterDescriptor, IKuromojiReadingFormTokenFilter> selector)
Parameters
Returns
|
Edit this page
View Source
KuromojiStemmer(string, Func<KuromojiStemmerTokenFilterDescriptor, IKuromojiStemmerTokenFilter>)
The kuromoji_stemmer token filter normalizes common katakana spelling variations ending in a
long sound character by removing this character (U+30FC). Only full-width katakana characters are supported.
Part of the analysis-kuromoji plugin:
Declaration
public TokenFiltersDescriptor KuromojiStemmer(string name, Func<KuromojiStemmerTokenFilterDescriptor, IKuromojiStemmerTokenFilter> selector = null)
Parameters
Returns
|
Edit this page
View Source
Length(string, Func<LengthTokenFilterDescriptor, ILengthTokenFilter>)
A token filter of type length that removes words that are too long or too short for the stream.
Declaration
public TokenFiltersDescriptor Length(string name, Func<LengthTokenFilterDescriptor, ILengthTokenFilter> selector)
Parameters
Returns
|
Edit this page
View Source
LimitTokenCount(string, Func<LimitTokenCountTokenFilterDescriptor, ILimitTokenCountTokenFilter>)
Limits the number of tokens that are indexed per document and field.
Declaration
public TokenFiltersDescriptor LimitTokenCount(string name, Func<LimitTokenCountTokenFilterDescriptor, ILimitTokenCountTokenFilter> selector)
Parameters
Returns
|
Edit this page
View Source
Lowercase(string, Func<LowercaseTokenFilterDescriptor, ILowercaseTokenFilter>)
A token filter of type lowercase that normalizes token text to lower case.
Lowercase token filter supports Greek and Turkish lowercase token filters through the language parameter.
Declaration
public TokenFiltersDescriptor Lowercase(string name, Func<LowercaseTokenFilterDescriptor, ILowercaseTokenFilter> selector = null)
Parameters
Returns
|
Edit this page
View Source
Multiplexer(string, Func<MultiplexerTokenFilterDescriptor, IMultiplexerTokenFilter>)
A token filter of type multiplexer will emit multiple tokens at the same position, each version of the token
having been run through a different filter. Identical output tokens at the same position will be removed.
Declaration
public TokenFiltersDescriptor Multiplexer(string name, Func<MultiplexerTokenFilterDescriptor, IMultiplexerTokenFilter> selector)
Parameters
Returns
|
Edit this page
View Source
NGram(string, Func<NGramTokenFilterDescriptor, INGramTokenFilter>)
A token filter of type nGram.
Declaration
public TokenFiltersDescriptor NGram(string name, Func<NGramTokenFilterDescriptor, INGramTokenFilter> selector)
Parameters
Returns
|
Edit this page
View Source
NoriPartOfSpeech(string, Func<NoriPartOfSpeechTokenFilterDescriptor, INoriPartOfSpeechTokenFilter>)
The nori_part_of_speech token filter removes tokens that match a set of part-of-speech tags.
Declaration
public TokenFiltersDescriptor NoriPartOfSpeech(string name, Func<NoriPartOfSpeechTokenFilterDescriptor, INoriPartOfSpeechTokenFilter> selector)
Parameters
Returns
|
Edit this page
View Source
PatternCapture(string, Func<PatternCaptureTokenFilterDescriptor, IPatternCaptureTokenFilter>)
The pattern_capture token filter, unlike the pattern tokenizer, emits a token for every capture group in the regular
expression.
Declaration
public TokenFiltersDescriptor PatternCapture(string name, Func<PatternCaptureTokenFilterDescriptor, IPatternCaptureTokenFilter> selector)
Parameters
Returns
|
Edit this page
View Source
PatternReplace(string, Func<PatternReplaceTokenFilterDescriptor, IPatternReplaceTokenFilter>)
The pattern_replace token filter allows to easily handle string replacements based on a regular expression.
Declaration
public TokenFiltersDescriptor PatternReplace(string name, Func<PatternReplaceTokenFilterDescriptor, IPatternReplaceTokenFilter> selector)
Parameters
Returns
|
Edit this page
View Source
Phonetic(string, Func<PhoneticTokenFilterDescriptor, IPhoneticTokenFilter>)
The phonetic token filter is provided as a plugin.
Declaration
public TokenFiltersDescriptor Phonetic(string name, Func<PhoneticTokenFilterDescriptor, IPhoneticTokenFilter> selector)
Parameters
Returns
|
Edit this page
View Source
PorterStem(string, Func<PorterStemTokenFilterDescriptor, IPorterStemTokenFilter>)
A token filter of type porterStem that transforms the token stream as per the Porter stemming algorithm.
Declaration
public TokenFiltersDescriptor PorterStem(string name, Func<PorterStemTokenFilterDescriptor, IPorterStemTokenFilter> selector = null)
Parameters
Returns
|
Edit this page
View Source
Predicate(string, Func<PredicateTokenFilterDescriptor, IPredicateTokenFilter>)
The predicate_token_filter token filter takes a predicate script, and removes tokens that do
not match the predicate.
Declaration
public TokenFiltersDescriptor Predicate(string name, Func<PredicateTokenFilterDescriptor, IPredicateTokenFilter> selector)
Parameters
Returns
|
Edit this page
View Source
RemoveDuplicates(string, Func<RemoveDuplicatesTokenFilterDescriptor, IRemoveDuplicatesTokenFilter>)
A token filter of type remove_duplicates that drops identical tokens at the same position.
Declaration
public TokenFiltersDescriptor RemoveDuplicates(string name, Func<RemoveDuplicatesTokenFilterDescriptor, IRemoveDuplicatesTokenFilter> selector = null)
Parameters
Returns
|
Edit this page
View Source
Reverse(string, Func<ReverseTokenFilterDescriptor, IReverseTokenFilter>)
A token filter of type reverse that simply reverses the tokens.
Declaration
public TokenFiltersDescriptor Reverse(string name, Func<ReverseTokenFilterDescriptor, IReverseTokenFilter> selector = null)
Parameters
Returns
|
Edit this page
View Source
Shingle(string, Func<ShingleTokenFilterDescriptor, IShingleTokenFilter>)
A token filter of type shingle that constructs shingles (token n-grams) from a token stream.
In other words, it creates combinations of tokens as a single token.
Declaration
public TokenFiltersDescriptor Shingle(string name, Func<ShingleTokenFilterDescriptor, IShingleTokenFilter> selector)
Parameters
Returns
|
Edit this page
View Source
Snowball(string, Func<SnowballTokenFilterDescriptor, ISnowballTokenFilter>)
A filter that stems words using a Snowball-generated stemmer.
Declaration
public TokenFiltersDescriptor Snowball(string name, Func<SnowballTokenFilterDescriptor, ISnowballTokenFilter> selector)
Parameters
Returns
|
Edit this page
View Source
Stemmer(string, Func<StemmerTokenFilterDescriptor, IStemmerTokenFilter>)
A filter that stems words (similar to snowball, but with more options).
Declaration
public TokenFiltersDescriptor Stemmer(string name, Func<StemmerTokenFilterDescriptor, IStemmerTokenFilter> selector)
Parameters
Returns
|
Edit this page
View Source
StemmerOverride(string, Func<StemmerOverrideTokenFilterDescriptor, IStemmerOverrideTokenFilter>)
Overrides stemming algorithms, by applying a custom mapping, then protecting these terms from being modified by
stemmers. Must be placed
before any stemming filters.
Declaration
public TokenFiltersDescriptor StemmerOverride(string name, Func<StemmerOverrideTokenFilterDescriptor, IStemmerOverrideTokenFilter> selector)
Parameters
Returns
|
Edit this page
View Source
Stop(string, Func<StopTokenFilterDescriptor, IStopTokenFilter>)
A token filter of type stop that removes stop words from token streams.
Declaration
public TokenFiltersDescriptor Stop(string name, Func<StopTokenFilterDescriptor, IStopTokenFilter> selector)
Parameters
Returns
|
Edit this page
View Source
Synonym(string, Func<SynonymTokenFilterDescriptor, ISynonymTokenFilter>)
The synonym token filter allows to easily handle synonyms during the analysis process.
Declaration
public TokenFiltersDescriptor Synonym(string name, Func<SynonymTokenFilterDescriptor, ISynonymTokenFilter> selector)
Parameters
Returns
|
Edit this page
View Source
SynonymGraph(string, Func<SynonymGraphTokenFilterDescriptor, ISynonymGraphTokenFilter>)
The synonym_graph token filter allows to easily handle synonyms,
including multi-word synonyms correctly during the analysis process.
Declaration
public TokenFiltersDescriptor SynonymGraph(string name, Func<SynonymGraphTokenFilterDescriptor, ISynonymGraphTokenFilter> selector)
Parameters
Returns
|
Edit this page
View Source
Trim(string, Func<TrimTokenFilterDescriptor, ITrimTokenFilter>)
The trim token filter trims surrounding whitespaces around a token.
Declaration
public TokenFiltersDescriptor Trim(string name, Func<TrimTokenFilterDescriptor, ITrimTokenFilter> selector = null)
Parameters
Returns
|
Edit this page
View Source
Truncate(string, Func<TruncateTokenFilterDescriptor, ITruncateTokenFilter>)
The truncate token filter can be used to truncate tokens into a specific length. This can come in handy with keyword
(single token)
based mapped fields that are used for sorting in order to reduce memory usage.
Declaration
public TokenFiltersDescriptor Truncate(string name, Func<TruncateTokenFilterDescriptor, ITruncateTokenFilter> selector)
Parameters
Returns
|
Edit this page
View Source
Unique(string, Func<UniqueTokenFilterDescriptor, IUniqueTokenFilter>)
The unique token filter can be used to only index unique tokens during analysis. By default it is applied on all the
token stream
Declaration
public TokenFiltersDescriptor Unique(string name, Func<UniqueTokenFilterDescriptor, IUniqueTokenFilter> selector = null)
Parameters
Returns
|
Edit this page
View Source
Uppercase(string, Func<UppercaseTokenFilterDescriptor, IUppercaseTokenFilter>)
A token filter of type uppercase that normalizes token text to upper case.
Declaration
public TokenFiltersDescriptor Uppercase(string name, Func<UppercaseTokenFilterDescriptor, IUppercaseTokenFilter> selector = null)
Parameters
Returns
|
Edit this page
View Source
UserDefined(string, ITokenFilter)
Declaration
public TokenFiltersDescriptor UserDefined(string name, ITokenFilter analyzer)
Parameters
Returns
|
Edit this page
View Source
WordDelimiter(string, Func<WordDelimiterTokenFilterDescriptor, IWordDelimiterTokenFilter>)
A token filter of type asciifolding that converts alphabetic, numeric, and symbolic Unicode characters which are
not in the first 127 ASCII characters (the “Basic Latin” Unicode block) into their ASCII equivalents, if one
exists.
Declaration
public TokenFiltersDescriptor WordDelimiter(string name, Func<WordDelimiterTokenFilterDescriptor, IWordDelimiterTokenFilter> selector)
Parameters
Returns
|
Edit this page
View Source
WordDelimiterGraph(string, Func<WordDelimiterGraphTokenFilterDescriptor, IWordDelimiterGraphTokenFilter>)
A token filter of type asciifolding that converts alphabetic, numeric, and symbolic Unicode characters which are
not in the first 127 ASCII characters (the “Basic Latin” Unicode block) into their ASCII equivalents, if one
exists.
Declaration
public TokenFiltersDescriptor WordDelimiterGraph(string name, Func<WordDelimiterGraphTokenFilterDescriptor, IWordDelimiterGraphTokenFilter> selector)
Parameters
Returns
Implements
Extension Methods