Interface IBulkAllRequest<T>

Namespace: OpenSearch.Client

Assembly: OpenSearch.Client.dll

Syntax

public interface IBulkAllRequest<T> where T : class

Type Parameters

Name	Description
T

Properties

BackOffRetries

In case of a HTTP 429 (Too Many Requests) response status code, how many times should we automatically back off before failing

Declaration

int? BackOffRetries { get; set; }

Property Value

Type	Description
int?

BackOffTime

In case of a HTTP 429 (Too Many Requests) response status code, how long should we wait before retrying

Declaration

Time BackOffTime { get; set; }

Property Value

Type	Description
Time

BackPressure

Simple back pressure implementation that makes sure the minimum max concurrency between producer and consumer is not amplified by the greedier of the two by more then a given back pressure factor When set each bulk request will call Release()

Declaration

ProducerConsumerBackPressure BackPressure { get; set; }

Property Value

Type	Description
ProducerConsumerBackPressure

BufferToBulk

By default, BulkAllObservable<T> calls IndexMany<T>(IEnumerable<T>, Func<BulkIndexDescriptor<T>, T, IBulkIndexOperation<T>>) on the buffer. There might be case where you'd like more control over the bulk operation. By setting this callback, you are in complete control of describing how the buffer should be translated to a bulk operation.

Declaration

Action<BulkDescriptor, IList<T>> BufferToBulk { get; set; }

Property Value

Type	Description
Action<BulkDescriptor, IList<T>>

BulkResponseCallback

Be notified every time a bulk response returns, this includes retries. OnNext(T) is only called for successful batches.

Declaration

Action<BulkResponse> BulkResponseCallback { get; set; }

Property Value

Type	Description
Action<BulkResponse>

ContinueAfterDroppedDocuments

Halt the bulk all request if any of the documents returned is a failure that can not be retried. When true, will feed dropped documents to DroppedDocumentCallback

Declaration

bool ContinueAfterDroppedDocuments { get; set; }

Property Value

Type	Description
bool

DocumentAffinityKey

Optional per-document affinity key. When set, documents sharing a key are routed to the same worker and that worker dispatches its batches strictly in order with at most one batch in flight, preserving on-the-wire ordering for operations that share a key. A slow or retrying worker holds up only its own key, not the others. When null (the default) batches are dispatched round-robin with no ordering guarantee.

Declaration

Func<T, string> DocumentAffinityKey { get; set; }

Property Value

Type	Description
Func<T, string>

DocumentId

Optional function that computes the _id of each document from the document itself, applied to the default bulk operation. When set, each document is indexed with Id(DocumentId(document)) instead of relying on id inference. The function can return any string — a single field, or a value computed from the document (for example d => $"{d.Region}-{d.Key}").

This is ignored when BufferToBulk is set: BufferToBulk gives you complete control over how the buffer is translated to a bulk operation, so you are responsible for setting the id yourself in that callback.

Declaration

Func<T, string> DocumentId { get; set; }

Property Value

Type	Description
Func<T, string>

Documents

The documents to send to OpenSearch, ideally lazily evaluated by using yield return to provide each document. BulkAllObservable<T> will eager evaluate each partitioned page when operating on it, using ToList<TSource>(IEnumerable<TSource>).

Declaration

IEnumerable<T> Documents { get; }

Property Value

Type	Description
IEnumerable<T>

DroppedDocumentCallback

If a bulk operation fails because it receives documents it can not retry they will be fed to this callback. If ContinueAfterDroppedDocuments is set to true processing will continue, so this callback can be used to feed into a dead letter queue. Otherwise bulk all indexing will be halted.

Declaration

Action<BulkResponseItemBase, T> DroppedDocumentCallback { get; set; }

Property Value

Type	Description
Action<BulkResponseItemBase, T>

Index

The index to use for items that don't specify one. By default, will be inferred from T. If no default index has been mapped for T using DefaultMappingFor<TDocument>(Func<ClrTypeMappingDescriptor<TDocument>, IClrTypeMapping<TDocument>>) on ConnectionSettings, an exception will be thrown.

Declaration

IndexName Index { get; set; }

Property Value

Type	Description
IndexName

MaxDegreeOfParallelism

The maximum number of bulk operations we want to have in flight at a time

Declaration

int? MaxDegreeOfParallelism { get; set; }

Property Value

Type	Description
int?

Pipeline

The pipeline id to preprocess all the incoming documents with

Declaration

string Pipeline { get; set; }

Property Value

Type	Description
string

RefreshIndices

The indices you wish to refresh after the bulk all completes, defaults to Index

Declaration

Indices RefreshIndices { get; set; }

Property Value

Type	Description
Indices

RefreshOnCompleted

Refresh the index after performing ALL the bulk operations (NOTE this is an additional request)

Declaration

bool RefreshOnCompleted { get; set; }

Property Value

Type	Description
bool

RetryBaseDelay

When set, retries use exponential backoff with jitter starting from this base delay (doubling per attempt, randomized, capped by RetryMaxDelay) instead of the fixed BackOffTime.

Declaration

TimeSpan? RetryBaseDelay { get; set; }

Property Value

Type	Description
TimeSpan?

RetryDocumentPredicate

A predicate to control which documents should be retried. Defaults to failed bulk items with a HTTP 429 (Too Many Requests) response status code.

Declaration

Func<BulkResponseItemBase, T, bool> RetryDocumentPredicate { get; set; }

Property Value

Type	Description
Func<BulkResponseItemBase, T, bool>

RetryMaxDelay

The maximum delay between retries when RetryBaseDelay is set. Ignored when RetryBaseDelay is not set (the fixed BackOffTime is used instead).

Declaration

TimeSpan? RetryMaxDelay { get; set; }

Property Value

Type	Description
TimeSpan?

Routing

Specific per bulk operation routing value

Declaration

Routing Routing { get; set; }

Property Value

Type	Description
Routing

Size

The number of documents to send per bulk

Declaration

int? Size { get; set; }

Property Value

Type	Description
int?

Timeout

Explicit per operation timeout

Declaration

Time Timeout { get; set; }

Property Value

Type	Description
Time

WaitForActiveShards

Sets the number of shard copies that must be active before proceeding with the bulk operation. Defaults to 1, meaning the primary shard only. Set to all for all shard copies, otherwise set to any non-negative value less than or equal to the total number of copies for the shard (number of replicas + 1)

Declaration

int? WaitForActiveShards { get; set; }

Property Value

Type	Description
int?

Extension Methods

SuffixExtensions.Suffix(object, string)