Compiler

Compilers are used to create keyvi dictionaries from key-value pairs. Different compiler types support different value types.

from keyvi.compiler import JsonDictionaryCompiler

compiler = JsonDictionaryCompiler({"memory_limit_mb": "256"})
compiler.add("key1", '{"value": 1}')
compiler.add("key2", '{"value": 2}')
compiler.compile()
compiler.write_to_file("data.kv")

Parameters

All compilers and mergers accept a params dictionary. Values are passed as strings.

Common compiler parameters:

Parameter

Type

Default

Description

memory_limit

int

1 GB

Memory limit in bytes. Also accepts memory_limit_kb, memory_limit_mb, memory_limit_gb.

temporary_path

str

system temp dir

Directory for temporary files during compilation.

minimization

bool

true

Enable FSA minimization. Disabling produces larger output but compiles faster.

parallel_sort_threshold

int

10000

Use parallel sort when key count exceeds this threshold. Set to 0 to disable.

Additional parameters for JsonDictionaryCompiler and FloatVectorDictionaryCompiler:

Parameter

Type

Default

Description

compression

str

none

Compression type: 'zlib', 'snappy', 'zstd', or empty for none.

compression_threshold

int

32

Minimum value size in bytes to apply compression.

floating_point_precision

str

double

JSON only. Set to 'single' for 32-bit floats.

vector_size

int

10

FloatVector only. Number of dimensions per vector.

Merger-specific parameters:

Parameter

Type

Default

Description

merge_mode

str

standard

Set to 'append' for append mode.

Compilers

class keyvi.compiler.JsonDictionaryCompiler

Compiler for creating a dictionary with JSON values.

Accepted params:

  • memory_limit (int): Memory limit in bytes (default: 1 GB). Also accepts memory_limit_kb, memory_limit_mb, memory_limit_gb.

  • temporary_path (str): Directory for temporary files (default: system temp directory).

  • minimization (bool): Enable FSA minimization (default: true).

  • parallel_sort_threshold (int): Threshold for parallel sort (default: 10000).

  • compression (str): Compression type: 'zlib', 'snappy', 'zstd', or none (default: none).

  • compression_threshold (int): Minimum value size in bytes to compress (default: 32).

  • floating_point_precision (str): Set to 'single' for 32-bit floats (default: double precision).

Add(self: keyvi._core.compiler.JsonDictionaryCompiler, key: str, value: str) → None
Compile(self: keyvi._core.compiler.JsonDictionaryCompiler, progress_callback: collections.abc.Callable[[int, int], None] = None) → None
WriteToFile(self: keyvi._core.compiler.JsonDictionaryCompiler, filename: str) → None
add(self: keyvi._core.compiler.JsonDictionaryCompiler, key: str, value: str) → None

Add a key-value pair.

compile(self: keyvi._core.compiler.JsonDictionaryCompiler, progress_callback: collections.abc.Callable[[int, int], None] = None) → None

Compile the dictionary. An optional progress callback receives (current, total) counts.

set_manifest(self: keyvi._core.compiler.JsonDictionaryCompiler, manifest: str) → None

Set custom metadata to be stored with the dictionary (e.g. version, build date).

write_to_file(self: keyvi._core.compiler.JsonDictionaryCompiler, filename: str) → None

Write the compiled dictionary to a file.

class keyvi.compiler.StringDictionaryCompiler

Compiler for creating a dictionary with string values.

Accepted params:

  • memory_limit (int): Memory limit in bytes (default: 1 GB). Also accepts memory_limit_kb, memory_limit_mb, memory_limit_gb.

  • temporary_path (str): Directory for temporary files (default: system temp directory).

  • minimization (bool): Enable FSA minimization (default: true).

  • parallel_sort_threshold (int): Threshold for parallel sort (default: 10000).

Add(self: keyvi._core.compiler.StringDictionaryCompiler, key: str, value: str) → None
Compile(self: keyvi._core.compiler.StringDictionaryCompiler, progress_callback: collections.abc.Callable[[int, int], None] = None) → None
WriteToFile(self: keyvi._core.compiler.StringDictionaryCompiler, filename: str) → None
add(self: keyvi._core.compiler.StringDictionaryCompiler, key: str, value: str) → None

Add a key-value pair.

compile(self: keyvi._core.compiler.StringDictionaryCompiler, progress_callback: collections.abc.Callable[[int, int], None] = None) → None

Compile the dictionary. An optional progress callback receives (current, total) counts.

set_manifest(self: keyvi._core.compiler.StringDictionaryCompiler, manifest: str) → None

Set custom metadata to be stored with the dictionary (e.g. version, build date).

write_to_file(self: keyvi._core.compiler.StringDictionaryCompiler, filename: str) → None

Write the compiled dictionary to a file.

class keyvi.compiler.IntDictionaryCompiler

Compiler for creating a dictionary with integer values.

Accepted params:

  • memory_limit (int): Memory limit in bytes (default: 1 GB). Also accepts memory_limit_kb, memory_limit_mb, memory_limit_gb.

  • temporary_path (str): Directory for temporary files (default: system temp directory).

  • minimization (bool): Enable FSA minimization (default: true).

  • parallel_sort_threshold (int): Threshold for parallel sort (default: 10000).

Add(self: keyvi._core.compiler.IntDictionaryCompiler, key: str, value: SupportsInt | SupportsIndex) → None
Compile(self: keyvi._core.compiler.IntDictionaryCompiler, progress_callback: collections.abc.Callable[[int, int], None] = None) → None
WriteToFile(self: keyvi._core.compiler.IntDictionaryCompiler, filename: str) → None
add(self: keyvi._core.compiler.IntDictionaryCompiler, key: str, value: SupportsInt | SupportsIndex) → None

Add a key-value pair.

compile(self: keyvi._core.compiler.IntDictionaryCompiler, progress_callback: collections.abc.Callable[[int, int], None] = None) → None

Compile the dictionary. An optional progress callback receives (current, total) counts.

set_manifest(self: keyvi._core.compiler.IntDictionaryCompiler, manifest: str) → None

Set custom metadata to be stored with the dictionary (e.g. version, build date).

write_to_file(self: keyvi._core.compiler.IntDictionaryCompiler, filename: str) → None

Write the compiled dictionary to a file.

class keyvi.compiler.CompletionDictionaryCompiler

Compiler for creating a completion dictionary with weighted string values.

Accepted params:

  • memory_limit (int): Memory limit in bytes (default: 1 GB). Also accepts memory_limit_kb, memory_limit_mb, memory_limit_gb.

  • temporary_path (str): Directory for temporary files (default: system temp directory).

  • minimization (bool): Enable FSA minimization (default: true).

  • parallel_sort_threshold (int): Threshold for parallel sort (default: 10000).

Add(self: keyvi._core.compiler.CompletionDictionaryCompiler, key: str, value: SupportsInt | SupportsIndex) → None
Compile(self: keyvi._core.compiler.CompletionDictionaryCompiler, progress_callback: collections.abc.Callable[[int, int], None] = None) → None
WriteToFile(self: keyvi._core.compiler.CompletionDictionaryCompiler, filename: str) → None
add(self: keyvi._core.compiler.CompletionDictionaryCompiler, key: str, value: SupportsInt | SupportsIndex) → None

Add a key-value pair.

compile(self: keyvi._core.compiler.CompletionDictionaryCompiler, progress_callback: collections.abc.Callable[[int, int], None] = None) → None

Compile the dictionary. An optional progress callback receives (current, total) counts.

set_manifest(self: keyvi._core.compiler.CompletionDictionaryCompiler, manifest: str) → None

Set custom metadata to be stored with the dictionary (e.g. version, build date).

write_to_file(self: keyvi._core.compiler.CompletionDictionaryCompiler, filename: str) → None

Write the compiled dictionary to a file.

class keyvi.compiler.FloatVectorDictionaryCompiler

Compiler for creating a dictionary with float vector values.

Accepted params:

  • memory_limit (int): Memory limit in bytes (default: 1 GB). Also accepts memory_limit_kb, memory_limit_mb, memory_limit_gb.

  • temporary_path (str): Directory for temporary files (default: system temp directory).

  • minimization (bool): Enable FSA minimization (default: true).

  • parallel_sort_threshold (int): Threshold for parallel sort (default: 10000).

  • compression (str): Compression type: 'zlib', 'snappy', 'zstd', or none (default: none).

  • compression_threshold (int): Minimum value size in bytes to compress (default: 32).

  • vector_size (int): Number of dimensions per vector (default: 10).

Add(self: keyvi._core.compiler.FloatVectorDictionaryCompiler, key: str, value: collections.abc.Sequence[SupportsFloat | SupportsIndex]) → None
Compile(self: keyvi._core.compiler.FloatVectorDictionaryCompiler, progress_callback: collections.abc.Callable[[int, int], None] = None) → None
WriteToFile(self: keyvi._core.compiler.FloatVectorDictionaryCompiler, filename: str) → None
add(self: keyvi._core.compiler.FloatVectorDictionaryCompiler, key: str, value: collections.abc.Sequence[SupportsFloat | SupportsIndex]) → None

Add a key-value pair.

compile(self: keyvi._core.compiler.FloatVectorDictionaryCompiler, progress_callback: collections.abc.Callable[[int, int], None] = None) → None

Compile the dictionary. An optional progress callback receives (current, total) counts.

set_manifest(self: keyvi._core.compiler.FloatVectorDictionaryCompiler, manifest: str) → None

Set custom metadata to be stored with the dictionary (e.g. version, build date).

write_to_file(self: keyvi._core.compiler.FloatVectorDictionaryCompiler, filename: str) → None

Write the compiled dictionary to a file.

class keyvi.compiler.KeyOnlyDictionaryCompiler

Compiler for creating a key-only keyvi dictionary.

Accepted params:

  • memory_limit (int): Memory limit in bytes (default: 1 GB). Also accepts memory_limit_kb, memory_limit_mb, memory_limit_gb.

  • temporary_path (str): Directory for temporary files (default: system temp directory).

  • minimization (bool): Enable FSA minimization (default: true).

  • parallel_sort_threshold (int): Threshold for parallel sort (default: 10000).

Add(self: keyvi._core.compiler.KeyOnlyDictionaryCompiler, key: str) → None
Compile(self: keyvi._core.compiler.KeyOnlyDictionaryCompiler, progress_callback: collections.abc.Callable[[int, int], None] = None) → None
WriteToFile(self: keyvi._core.compiler.KeyOnlyDictionaryCompiler, filename: str) → None
add(self: keyvi._core.compiler.KeyOnlyDictionaryCompiler, key: str) → None

Add a key.

compile(self: keyvi._core.compiler.KeyOnlyDictionaryCompiler, progress_callback: collections.abc.Callable[[int, int], None] = None) → None

Compile the dictionary. An optional progress callback receives (current, total) counts.

set_manifest(self: keyvi._core.compiler.KeyOnlyDictionaryCompiler, manifest: str) → None

Set custom metadata to be stored with the dictionary (e.g. version, build date).

write_to_file(self: keyvi._core.compiler.KeyOnlyDictionaryCompiler, filename: str) → None

Write the compiled dictionary to a file.

Secondary Key Compilers

class keyvi.compiler.SecondaryKeyJsonDictionaryCompiler

Compiler for creating a secondary key dictionary with JSON values.

Accepted params:

  • memory_limit (int): Memory limit in bytes (default: 1 GB). Also accepts memory_limit_kb, memory_limit_mb, memory_limit_gb.

  • temporary_path (str): Directory for temporary files (default: system temp directory).

  • minimization (bool): Enable FSA minimization (default: true).

  • parallel_sort_threshold (int): Threshold for parallel sort (default: 10000).

  • compression (str): Compression type: 'zlib', 'snappy', 'zstd', or none (default: none).

  • compression_threshold (int): Minimum value size in bytes to compress (default: 32).

  • floating_point_precision (str): Set to 'single' for 32-bit floats (default: double precision).

Compile(self: keyvi._core.compiler.SecondaryKeyJsonDictionaryCompiler, progress_callback: collections.abc.Callable[[int, int], None] = None) → None
WriteToFile(self: keyvi._core.compiler.SecondaryKeyJsonDictionaryCompiler, filename: str) → None
add(self: keyvi._core.compiler.SecondaryKeyJsonDictionaryCompiler, arg0: str, arg1: collections.abc.Mapping[str, str], arg2: str) → None

Add a key-value pair with secondary keys.

compile(self: keyvi._core.compiler.SecondaryKeyJsonDictionaryCompiler, progress_callback: collections.abc.Callable[[int, int], None] = None) → None

Compile the dictionary. An optional progress callback receives (current, total) counts.

set_manifest(self: keyvi._core.compiler.SecondaryKeyJsonDictionaryCompiler, manifest: str) → None

Set custom metadata to be stored with the dictionary (e.g. version, build date).

write_to_file(self: keyvi._core.compiler.SecondaryKeyJsonDictionaryCompiler, filename: str) → None

Write the compiled dictionary to a file.

class keyvi.compiler.SecondaryKeyStringDictionaryCompiler

Compiler for creating a secondary key dictionary with string values.

Accepted params:

  • memory_limit (int): Memory limit in bytes (default: 1 GB). Also accepts memory_limit_kb, memory_limit_mb, memory_limit_gb.

  • temporary_path (str): Directory for temporary files (default: system temp directory).

  • minimization (bool): Enable FSA minimization (default: true).

  • parallel_sort_threshold (int): Threshold for parallel sort (default: 10000).

Compile(self: keyvi._core.compiler.SecondaryKeyStringDictionaryCompiler, progress_callback: collections.abc.Callable[[int, int], None] = None) → None
WriteToFile(self: keyvi._core.compiler.SecondaryKeyStringDictionaryCompiler, filename: str) → None
add(self: keyvi._core.compiler.SecondaryKeyStringDictionaryCompiler, arg0: str, arg1: collections.abc.Mapping[str, str], arg2: str) → None

Add a key-value pair with secondary keys.

compile(self: keyvi._core.compiler.SecondaryKeyStringDictionaryCompiler, progress_callback: collections.abc.Callable[[int, int], None] = None) → None

Compile the dictionary. An optional progress callback receives (current, total) counts.

set_manifest(self: keyvi._core.compiler.SecondaryKeyStringDictionaryCompiler, manifest: str) → None

Set custom metadata to be stored with the dictionary (e.g. version, build date).

write_to_file(self: keyvi._core.compiler.SecondaryKeyStringDictionaryCompiler, filename: str) → None

Write the compiled dictionary to a file.

class keyvi.compiler.SecondaryKeyIntDictionaryCompiler

Compiler for creating a secondary key dictionary with integer values.

Accepted params:

  • memory_limit (int): Memory limit in bytes (default: 1 GB). Also accepts memory_limit_kb, memory_limit_mb, memory_limit_gb.

  • temporary_path (str): Directory for temporary files (default: system temp directory).

  • minimization (bool): Enable FSA minimization (default: true).

  • parallel_sort_threshold (int): Threshold for parallel sort (default: 10000).

Compile(self: keyvi._core.compiler.SecondaryKeyIntDictionaryCompiler, progress_callback: collections.abc.Callable[[int, int], None] = None) → None
WriteToFile(self: keyvi._core.compiler.SecondaryKeyIntDictionaryCompiler, filename: str) → None
add(self: keyvi._core.compiler.SecondaryKeyIntDictionaryCompiler, arg0: str, arg1: collections.abc.Mapping[str, str], arg2: SupportsInt | SupportsIndex) → None

Add a key-value pair with secondary keys.

compile(self: keyvi._core.compiler.SecondaryKeyIntDictionaryCompiler, progress_callback: collections.abc.Callable[[int, int], None] = None) → None

Compile the dictionary. An optional progress callback receives (current, total) counts.

set_manifest(self: keyvi._core.compiler.SecondaryKeyIntDictionaryCompiler, manifest: str) → None

Set custom metadata to be stored with the dictionary (e.g. version, build date).

write_to_file(self: keyvi._core.compiler.SecondaryKeyIntDictionaryCompiler, filename: str) → None

Write the compiled dictionary to a file.

class keyvi.compiler.SecondaryKeyCompletionDictionaryCompiler

Compiler for creating a secondary key dictionary with weighted string values.

Accepted params:

  • memory_limit (int): Memory limit in bytes (default: 1 GB). Also accepts memory_limit_kb, memory_limit_mb, memory_limit_gb.

  • temporary_path (str): Directory for temporary files (default: system temp directory).

  • minimization (bool): Enable FSA minimization (default: true).

  • parallel_sort_threshold (int): Threshold for parallel sort (default: 10000).

Compile(self: keyvi._core.compiler.SecondaryKeyCompletionDictionaryCompiler, progress_callback: collections.abc.Callable[[int, int], None] = None) → None
WriteToFile(self: keyvi._core.compiler.SecondaryKeyCompletionDictionaryCompiler, filename: str) → None
add(self: keyvi._core.compiler.SecondaryKeyCompletionDictionaryCompiler, arg0: str, arg1: collections.abc.Mapping[str, str], arg2: SupportsInt | SupportsIndex) → None

Add a key-value pair with secondary keys.

compile(self: keyvi._core.compiler.SecondaryKeyCompletionDictionaryCompiler, progress_callback: collections.abc.Callable[[int, int], None] = None) → None

Compile the dictionary. An optional progress callback receives (current, total) counts.

set_manifest(self: keyvi._core.compiler.SecondaryKeyCompletionDictionaryCompiler, manifest: str) → None

Set custom metadata to be stored with the dictionary (e.g. version, build date).

write_to_file(self: keyvi._core.compiler.SecondaryKeyCompletionDictionaryCompiler, filename: str) → None

Write the compiled dictionary to a file.

class keyvi.compiler.SecondaryKeyFloatVectorDictionaryCompiler

Compiler for creating a secondary key dictionary with float vector values.

Accepted params:

  • memory_limit (int): Memory limit in bytes (default: 1 GB). Also accepts memory_limit_kb, memory_limit_mb, memory_limit_gb.

  • temporary_path (str): Directory for temporary files (default: system temp directory).

  • minimization (bool): Enable FSA minimization (default: true).

  • parallel_sort_threshold (int): Threshold for parallel sort (default: 10000).

  • compression (str): Compression type: 'zlib', 'snappy', 'zstd', or none (default: none).

  • compression_threshold (int): Minimum value size in bytes to compress (default: 32).

  • vector_size (int): Number of dimensions per vector (default: 10).

Compile(self: keyvi._core.compiler.SecondaryKeyFloatVectorDictionaryCompiler, progress_callback: collections.abc.Callable[[int, int], None] = None) → None
WriteToFile(self: keyvi._core.compiler.SecondaryKeyFloatVectorDictionaryCompiler, filename: str) → None
add(self: keyvi._core.compiler.SecondaryKeyFloatVectorDictionaryCompiler, arg0: str, arg1: collections.abc.Mapping[str, str], arg2: collections.abc.Sequence[SupportsFloat | SupportsIndex]) → None

Add a key-value pair with secondary keys.

compile(self: keyvi._core.compiler.SecondaryKeyFloatVectorDictionaryCompiler, progress_callback: collections.abc.Callable[[int, int], None] = None) → None

Compile the dictionary. An optional progress callback receives (current, total) counts.

set_manifest(self: keyvi._core.compiler.SecondaryKeyFloatVectorDictionaryCompiler, manifest: str) → None

Set custom metadata to be stored with the dictionary (e.g. version, build date).

write_to_file(self: keyvi._core.compiler.SecondaryKeyFloatVectorDictionaryCompiler, filename: str) → None

Write the compiled dictionary to a file.

class keyvi.compiler.SecondaryKeyKeyOnlyDictionaryCompiler

Compiler for creating a secondary key dictionary with keys only (no values).

Accepted params:

  • memory_limit (int): Memory limit in bytes (default: 1 GB). Also accepts memory_limit_kb, memory_limit_mb, memory_limit_gb.

  • temporary_path (str): Directory for temporary files (default: system temp directory).

  • minimization (bool): Enable FSA minimization (default: true).

  • parallel_sort_threshold (int): Threshold for parallel sort (default: 10000).

Compile(self: keyvi._core.compiler.SecondaryKeyKeyOnlyDictionaryCompiler, progress_callback: collections.abc.Callable[[int, int], None] = None) → None
WriteToFile(self: keyvi._core.compiler.SecondaryKeyKeyOnlyDictionaryCompiler, filename: str) → None
add(self: keyvi._core.compiler.SecondaryKeyKeyOnlyDictionaryCompiler, arg0: str, arg1: collections.abc.Mapping[str, str], arg2: SupportsInt | SupportsIndex) → None

Add a key-value pair with secondary keys.

compile(self: keyvi._core.compiler.SecondaryKeyKeyOnlyDictionaryCompiler, progress_callback: collections.abc.Callable[[int, int], None] = None) → None

Compile the dictionary. An optional progress callback receives (current, total) counts.

set_manifest(self: keyvi._core.compiler.SecondaryKeyKeyOnlyDictionaryCompiler, manifest: str) → None

Set custom metadata to be stored with the dictionary (e.g. version, build date).

write_to_file(self: keyvi._core.compiler.SecondaryKeyKeyOnlyDictionaryCompiler, filename: str) → None

Write the compiled dictionary to a file.

Mergers

Mergers combine multiple keyvi dictionaries into one.

class keyvi.compiler.JsonDictionaryMerger

Merger for combining multiple JSON dictionaries into one.

Accepted params:

  • memory_limit (int): Memory limit in bytes (default: 1 GB). Also accepts memory_limit_kb, memory_limit_mb, memory_limit_gb.

  • temporary_path (str): Directory for temporary files (default: system temp directory).

  • merge_mode (str): Set to 'append' for append mode (default: standard merge).

add(self: keyvi._core.compiler.JsonDictionaryMerger, filename: str) → None

Add a dictionary file to be merged.

merge(*args, **kwargs)

Overloaded function.

  1. merge(self: keyvi._core.compiler.JsonDictionaryMerger) -> None

Merge all added dictionaries. The result can be written with write_to_file.

  1. merge(self: keyvi._core.compiler.JsonDictionaryMerger, filename: str) -> None

Merge all added dictionaries and write the result to the given file.

set_manifest(self: keyvi._core.compiler.JsonDictionaryMerger, manifest: str) → None

Set custom metadata to be stored with the merged dictionary (e.g. version, build date).

write_to_file(self: keyvi._core.compiler.JsonDictionaryMerger, filename: str) → None

Write the merged dictionary to a file.

class keyvi.compiler.StringDictionaryMerger

Merger for combining multiple string dictionaries into one.

Accepted params:

  • memory_limit (int): Memory limit in bytes (default: 1 GB). Also accepts memory_limit_kb, memory_limit_mb, memory_limit_gb.

  • temporary_path (str): Directory for temporary files (default: system temp directory).

  • merge_mode (str): Set to 'append' for append mode (default: standard merge).

add(self: keyvi._core.compiler.StringDictionaryMerger, filename: str) → None

Add a dictionary file to be merged.

merge(*args, **kwargs)

Overloaded function.

  1. merge(self: keyvi._core.compiler.StringDictionaryMerger) -> None

Merge all added dictionaries. The result can be written with write_to_file.

  1. merge(self: keyvi._core.compiler.StringDictionaryMerger, filename: str) -> None

Merge all added dictionaries and write the result to the given file.

set_manifest(self: keyvi._core.compiler.StringDictionaryMerger, manifest: str) → None

Set custom metadata to be stored with the merged dictionary (e.g. version, build date).

write_to_file(self: keyvi._core.compiler.StringDictionaryMerger, filename: str) → None

Write the merged dictionary to a file.

class keyvi.compiler.IntDictionaryMerger

Merger for combining multiple integer dictionaries into one.

Accepted params:

  • memory_limit (int): Memory limit in bytes (default: 1 GB). Also accepts memory_limit_kb, memory_limit_mb, memory_limit_gb.

  • temporary_path (str): Directory for temporary files (default: system temp directory).

  • merge_mode (str): Set to 'append' for append mode (default: standard merge).

add(self: keyvi._core.compiler.IntDictionaryMerger, filename: str) → None

Add a dictionary file to be merged.

merge(*args, **kwargs)

Overloaded function.

  1. merge(self: keyvi._core.compiler.IntDictionaryMerger) -> None

Merge all added dictionaries. The result can be written with write_to_file.

  1. merge(self: keyvi._core.compiler.IntDictionaryMerger, filename: str) -> None

Merge all added dictionaries and write the result to the given file.

set_manifest(self: keyvi._core.compiler.IntDictionaryMerger, manifest: str) → None

Set custom metadata to be stored with the merged dictionary (e.g. version, build date).

write_to_file(self: keyvi._core.compiler.IntDictionaryMerger, filename: str) → None

Write the merged dictionary to a file.

class keyvi.compiler.CompletionDictionaryMerger

Merger for combining multiple completion dictionaries into one.

Accepted params:

  • memory_limit (int): Memory limit in bytes (default: 1 GB). Also accepts memory_limit_kb, memory_limit_mb, memory_limit_gb.

  • temporary_path (str): Directory for temporary files (default: system temp directory).

  • merge_mode (str): Set to 'append' for append mode (default: standard merge).

add(self: keyvi._core.compiler.CompletionDictionaryMerger, filename: str) → None

Add a dictionary file to be merged.

merge(*args, **kwargs)

Overloaded function.

  1. merge(self: keyvi._core.compiler.CompletionDictionaryMerger) -> None

Merge all added dictionaries. The result can be written with write_to_file.

  1. merge(self: keyvi._core.compiler.CompletionDictionaryMerger, filename: str) -> None

Merge all added dictionaries and write the result to the given file.

set_manifest(self: keyvi._core.compiler.CompletionDictionaryMerger, manifest: str) → None

Set custom metadata to be stored with the merged dictionary (e.g. version, build date).

write_to_file(self: keyvi._core.compiler.CompletionDictionaryMerger, filename: str) → None

Write the merged dictionary to a file.

class keyvi.compiler.KeyOnlyDictionaryMerger

Merger for combining multiple key-only dictionaries into one.

Accepted params:

  • memory_limit (int): Memory limit in bytes (default: 1 GB). Also accepts memory_limit_kb, memory_limit_mb, memory_limit_gb.

  • temporary_path (str): Directory for temporary files (default: system temp directory).

  • merge_mode (str): Set to 'append' for append mode (default: standard merge).

add(self: keyvi._core.compiler.KeyOnlyDictionaryMerger, filename: str) → None

Add a dictionary file to be merged.

merge(*args, **kwargs)

Overloaded function.

  1. merge(self: keyvi._core.compiler.KeyOnlyDictionaryMerger) -> None

Merge all added dictionaries. The result can be written with write_to_file.

  1. merge(self: keyvi._core.compiler.KeyOnlyDictionaryMerger, filename: str) -> None

Merge all added dictionaries and write the result to the given file.

set_manifest(self: keyvi._core.compiler.KeyOnlyDictionaryMerger, manifest: str) → None

Set custom metadata to be stored with the merged dictionary (e.g. version, build date).

write_to_file(self: keyvi._core.compiler.KeyOnlyDictionaryMerger, filename: str) → None

Write the merged dictionary to a file.

Generator

class keyvi.compiler.KeyOnlyDictionaryGenerator

Generator for creating a key-only dictionary with pre-sorted keys.

add(self: keyvi._core.compiler.KeyOnlyDictionaryGenerator, key: str) → None

Add a key. Keys must be added in sorted order.

close_feeding(self: keyvi._core.compiler.KeyOnlyDictionaryGenerator) → None

Signal that no more keys will be added.

write_to_file(self: keyvi._core.compiler.KeyOnlyDictionaryGenerator, filename: str) → None

Write the dictionary to a file.