Skip to content

Conversion

Conversion Manual

Overview

The 'conversion' codecs are a family of codecs that convert the type of the input from its original type to the target type to convert to. There is a conversion codec from the serial type to any other type, and a conversion codec from any other type to the serial type.

Inputs

SerialToToken, SerialToNumeric, SerialToString, TokenToSerial, NumToSerial

An input of type serial, struct, numeric.

StringToSerial

  • An input of type string
  • A numeric input containing the lengths of the strings.

Outputs

SerialToToken, SerialToNumeric, StringToSerial, TokenToSerial, NumToSerial

A single output of the specified type that is compatible - serial, struct, numeric. The output contains the converted data in the output format. See the spec for more details on the typed format.

SerialToString

StringToSerial has 2 ouputs, one of type serial containing a serial representation of the string data, the second output contains a numeric output containing sizes of the strings in the input.

Use Cases

Used to invoke codecs with different underlying types from the received type. One common example is entropy codecs which only take serial inputs. When compressing numeric data, it is necessary to convert to serial then run the entropy codecs.

openzl.ext.nodes.ConvertSerialToNumLE

Bases: Node

Source code in build-openzl/py/site-packages/openzl/ext/nodes.pyi
class ConvertSerialToNumLE(Node):
    def __init__(self, int_size_bytes: int) -> None: ...

    def __call__(self, compressor: ext.Compressor, successor: ext.GraphID | ext.graphs.Graph) -> ext.GraphID: ...

    def run(self, edge: ext.Edge) -> list[ext.Edge]: ...

    def run_multi_input(self, edges: Sequence[ext.Edge]) -> list[ext.Edge]: ...

    def build_graph(self, compressor: ext.Compressor, successors: Sequence[ext.GraphID]) -> ext.GraphID: ...

    def parameterize(self, compressor: ext.Compressor) -> ext.NodeID: ...

    @property
    def base_node(self) -> ext.NodeID: ...

base_node property

__call__(compressor, successor)

Source code in build-openzl/py/site-packages/openzl/ext/nodes.pyi
def __call__(self, compressor: ext.Compressor, successor: ext.GraphID | ext.graphs.Graph) -> ext.GraphID: ...

__init__(int_size_bytes)

Source code in build-openzl/py/site-packages/openzl/ext/nodes.pyi
def __init__(self, int_size_bytes: int) -> None: ...

build_graph(compressor, successors)

Source code in build-openzl/py/site-packages/openzl/ext/nodes.pyi
def build_graph(self, compressor: ext.Compressor, successors: Sequence[ext.GraphID]) -> ext.GraphID: ...

parameterize(compressor)

Source code in build-openzl/py/site-packages/openzl/ext/nodes.pyi
def parameterize(self, compressor: ext.Compressor) -> ext.NodeID: ...

run(edge)

Source code in build-openzl/py/site-packages/openzl/ext/nodes.pyi
def run(self, edge: ext.Edge) -> list[ext.Edge]: ...

run_multi_input(edges)

Source code in build-openzl/py/site-packages/openzl/ext/nodes.pyi
def run_multi_input(self, edges: Sequence[ext.Edge]) -> list[ext.Edge]: ...

openzl.ext.nodes.ConvertSerialToNumBE

Bases: Node

Source code in build-openzl/py/site-packages/openzl/ext/nodes.pyi
class ConvertSerialToNumBE(Node):
    def __init__(self, int_size_bytes: int) -> None: ...

    def __call__(self, compressor: ext.Compressor, successor: ext.GraphID | ext.graphs.Graph) -> ext.GraphID: ...

    def run(self, edge: ext.Edge) -> list[ext.Edge]: ...

    def run_multi_input(self, edges: Sequence[ext.Edge]) -> list[ext.Edge]: ...

    def build_graph(self, compressor: ext.Compressor, successors: Sequence[ext.GraphID]) -> ext.GraphID: ...

    def parameterize(self, compressor: ext.Compressor) -> ext.NodeID: ...

    @property
    def base_node(self) -> ext.NodeID: ...

base_node property

__call__(compressor, successor)

Source code in build-openzl/py/site-packages/openzl/ext/nodes.pyi
def __call__(self, compressor: ext.Compressor, successor: ext.GraphID | ext.graphs.Graph) -> ext.GraphID: ...

__init__(int_size_bytes)

Source code in build-openzl/py/site-packages/openzl/ext/nodes.pyi
def __init__(self, int_size_bytes: int) -> None: ...

build_graph(compressor, successors)

Source code in build-openzl/py/site-packages/openzl/ext/nodes.pyi
def build_graph(self, compressor: ext.Compressor, successors: Sequence[ext.GraphID]) -> ext.GraphID: ...

parameterize(compressor)

Source code in build-openzl/py/site-packages/openzl/ext/nodes.pyi
def parameterize(self, compressor: ext.Compressor) -> ext.NodeID: ...

run(edge)

Source code in build-openzl/py/site-packages/openzl/ext/nodes.pyi
def run(self, edge: ext.Edge) -> list[ext.Edge]: ...

run_multi_input(edges)

Source code in build-openzl/py/site-packages/openzl/ext/nodes.pyi
def run_multi_input(self, edges: Sequence[ext.Edge]) -> list[ext.Edge]: ...

openzl.ext.nodes.ConvertSerialToStruct

Bases: Node

Convert a serial input to a struct output with the given struct size

Inputs: input: Type.Serial

Singleton Outputs: converted: Type.Struct

Source code in build-openzl/py/site-packages/openzl/ext/nodes.pyi
class ConvertSerialToStruct(Node):
    """
    Convert a serial input to a struct output with the given struct size

    Inputs:
    input: Type.Serial


    Singleton Outputs:
    converted: Type.Struct
    """

    def __init__(self, struct_size_bytes: int) -> None: ...

    def __call__(self, compressor: ext.Compressor, successor: ext.GraphID | ext.graphs.Graph) -> ext.GraphID: ...

    def run(self, edge: ext.Edge) -> list[ext.Edge]: ...

    def run_multi_input(self, edges: Sequence[ext.Edge]) -> list[ext.Edge]: ...

    def build_graph(self, compressor: ext.Compressor, successors: Sequence[ext.GraphID]) -> ext.GraphID: ...

    def parameterize(self, compressor: ext.Compressor) -> ext.NodeID: ...

    @property
    def base_node(self) -> ext.NodeID: ...

base_node property

__call__(compressor, successor)

Source code in build-openzl/py/site-packages/openzl/ext/nodes.pyi
def __call__(self, compressor: ext.Compressor, successor: ext.GraphID | ext.graphs.Graph) -> ext.GraphID: ...

__init__(struct_size_bytes)

Source code in build-openzl/py/site-packages/openzl/ext/nodes.pyi
def __init__(self, struct_size_bytes: int) -> None: ...

build_graph(compressor, successors)

Source code in build-openzl/py/site-packages/openzl/ext/nodes.pyi
def build_graph(self, compressor: ext.Compressor, successors: Sequence[ext.GraphID]) -> ext.GraphID: ...

parameterize(compressor)

Source code in build-openzl/py/site-packages/openzl/ext/nodes.pyi
def parameterize(self, compressor: ext.Compressor) -> ext.NodeID: ...

run(edge)

Source code in build-openzl/py/site-packages/openzl/ext/nodes.pyi
def run(self, edge: ext.Edge) -> list[ext.Edge]: ...

run_multi_input(edges)

Source code in build-openzl/py/site-packages/openzl/ext/nodes.pyi
def run_multi_input(self, edges: Sequence[ext.Edge]) -> list[ext.Edge]: ...

openzl.ext.nodes.ConvertStructToNumLE

Bases: Node

Convert little-endian fixed-size struct input to numeric output

Inputs: input: Type.Struct

Singleton Outputs: converted: Type.Numeric

Source code in build-openzl/py/site-packages/openzl/ext/nodes.pyi
class ConvertStructToNumLE(Node):
    """
    Convert little-endian fixed-size struct input to numeric output

    Inputs:
    input: Type.Struct


    Singleton Outputs:
    converted: Type.Numeric
    """

    def __init__(self) -> None: ...

    def __call__(self, compressor: ext.Compressor, successor: ext.GraphID | ext.graphs.Graph) -> ext.GraphID: ...

    def run(self, edge: ext.Edge) -> list[ext.Edge]: ...

    def run_multi_input(self, edges: Sequence[ext.Edge]) -> list[ext.Edge]: ...

    def build_graph(self, compressor: ext.Compressor, successors: Sequence[ext.GraphID]) -> ext.GraphID: ...

    def parameterize(self, compressor: ext.Compressor) -> ext.NodeID: ...

    @property
    def base_node(self) -> ext.NodeID: ...

base_node property

__call__(compressor, successor)

Source code in build-openzl/py/site-packages/openzl/ext/nodes.pyi
def __call__(self, compressor: ext.Compressor, successor: ext.GraphID | ext.graphs.Graph) -> ext.GraphID: ...

__init__()

Source code in build-openzl/py/site-packages/openzl/ext/nodes.pyi
def __init__(self) -> None: ...

build_graph(compressor, successors)

Source code in build-openzl/py/site-packages/openzl/ext/nodes.pyi
def build_graph(self, compressor: ext.Compressor, successors: Sequence[ext.GraphID]) -> ext.GraphID: ...

parameterize(compressor)

Source code in build-openzl/py/site-packages/openzl/ext/nodes.pyi
def parameterize(self, compressor: ext.Compressor) -> ext.NodeID: ...

run(edge)

Source code in build-openzl/py/site-packages/openzl/ext/nodes.pyi
def run(self, edge: ext.Edge) -> list[ext.Edge]: ...

run_multi_input(edges)

Source code in build-openzl/py/site-packages/openzl/ext/nodes.pyi
def run_multi_input(self, edges: Sequence[ext.Edge]) -> list[ext.Edge]: ...

openzl.ext.nodes.ConvertStructToNumBE

Bases: Node

Convert big-endian fixed-size struct input to numeric output

Inputs: input: Type.Struct

Singleton Outputs: converted: Type.Numeric

Source code in build-openzl/py/site-packages/openzl/ext/nodes.pyi
class ConvertStructToNumBE(Node):
    """
    Convert big-endian fixed-size struct input to numeric output

    Inputs:
    input: Type.Struct


    Singleton Outputs:
    converted: Type.Numeric
    """

    def __init__(self) -> None: ...

    def __call__(self, compressor: ext.Compressor, successor: ext.GraphID | ext.graphs.Graph) -> ext.GraphID: ...

    def run(self, edge: ext.Edge) -> list[ext.Edge]: ...

    def run_multi_input(self, edges: Sequence[ext.Edge]) -> list[ext.Edge]: ...

    def build_graph(self, compressor: ext.Compressor, successors: Sequence[ext.GraphID]) -> ext.GraphID: ...

    def parameterize(self, compressor: ext.Compressor) -> ext.NodeID: ...

    @property
    def base_node(self) -> ext.NodeID: ...

base_node property

__call__(compressor, successor)

Source code in build-openzl/py/site-packages/openzl/ext/nodes.pyi
def __call__(self, compressor: ext.Compressor, successor: ext.GraphID | ext.graphs.Graph) -> ext.GraphID: ...

__init__()

Source code in build-openzl/py/site-packages/openzl/ext/nodes.pyi
def __init__(self) -> None: ...

build_graph(compressor, successors)

Source code in build-openzl/py/site-packages/openzl/ext/nodes.pyi
def build_graph(self, compressor: ext.Compressor, successors: Sequence[ext.GraphID]) -> ext.GraphID: ...

parameterize(compressor)

Source code in build-openzl/py/site-packages/openzl/ext/nodes.pyi
def parameterize(self, compressor: ext.Compressor) -> ext.NodeID: ...

run(edge)

Source code in build-openzl/py/site-packages/openzl/ext/nodes.pyi
def run(self, edge: ext.Edge) -> list[ext.Edge]: ...

run_multi_input(edges)

Source code in build-openzl/py/site-packages/openzl/ext/nodes.pyi
def run_multi_input(self, edges: Sequence[ext.Edge]) -> list[ext.Edge]: ...

openzl.ext.nodes.ConvertNumToSerialLE

Bases: Node

Convert numeric to serial in little-endian format

Inputs: input: Type.Numeric

Singleton Outputs: converted: Type.Serial

Source code in build-openzl/py/site-packages/openzl/ext/nodes.pyi
class ConvertNumToSerialLE(Node):
    """
    Convert numeric to serial in little-endian format

    Inputs:
    input: Type.Numeric


    Singleton Outputs:
    converted: Type.Serial
    """

    def __init__(self) -> None: ...

    def __call__(self, compressor: ext.Compressor, successor: ext.GraphID | ext.graphs.Graph) -> ext.GraphID: ...

    def run(self, edge: ext.Edge) -> list[ext.Edge]: ...

    def run_multi_input(self, edges: Sequence[ext.Edge]) -> list[ext.Edge]: ...

    def build_graph(self, compressor: ext.Compressor, successors: Sequence[ext.GraphID]) -> ext.GraphID: ...

    def parameterize(self, compressor: ext.Compressor) -> ext.NodeID: ...

    @property
    def base_node(self) -> ext.NodeID: ...

base_node property

__call__(compressor, successor)

Source code in build-openzl/py/site-packages/openzl/ext/nodes.pyi
def __call__(self, compressor: ext.Compressor, successor: ext.GraphID | ext.graphs.Graph) -> ext.GraphID: ...

__init__()

Source code in build-openzl/py/site-packages/openzl/ext/nodes.pyi
def __init__(self) -> None: ...

build_graph(compressor, successors)

Source code in build-openzl/py/site-packages/openzl/ext/nodes.pyi
def build_graph(self, compressor: ext.Compressor, successors: Sequence[ext.GraphID]) -> ext.GraphID: ...

parameterize(compressor)

Source code in build-openzl/py/site-packages/openzl/ext/nodes.pyi
def parameterize(self, compressor: ext.Compressor) -> ext.NodeID: ...

run(edge)

Source code in build-openzl/py/site-packages/openzl/ext/nodes.pyi
def run(self, edge: ext.Edge) -> list[ext.Edge]: ...

run_multi_input(edges)

Source code in build-openzl/py/site-packages/openzl/ext/nodes.pyi
def run_multi_input(self, edges: Sequence[ext.Edge]) -> list[ext.Edge]: ...

openzl.ext.nodes.ConvertNumToStructLE

Bases: Node

Convert numeric input to a little-endian fixed-size struct output

Inputs: input: Type.Numeric

Singleton Outputs: converted: Type.Struct

Source code in build-openzl/py/site-packages/openzl/ext/nodes.pyi
class ConvertNumToStructLE(Node):
    """
    Convert numeric input to a little-endian fixed-size struct output

    Inputs:
    input: Type.Numeric


    Singleton Outputs:
    converted: Type.Struct
    """

    def __init__(self) -> None: ...

    def __call__(self, compressor: ext.Compressor, successor: ext.GraphID | ext.graphs.Graph) -> ext.GraphID: ...

    def run(self, edge: ext.Edge) -> list[ext.Edge]: ...

    def run_multi_input(self, edges: Sequence[ext.Edge]) -> list[ext.Edge]: ...

    def build_graph(self, compressor: ext.Compressor, successors: Sequence[ext.GraphID]) -> ext.GraphID: ...

    def parameterize(self, compressor: ext.Compressor) -> ext.NodeID: ...

    @property
    def base_node(self) -> ext.NodeID: ...

base_node property

__call__(compressor, successor)

Source code in build-openzl/py/site-packages/openzl/ext/nodes.pyi
def __call__(self, compressor: ext.Compressor, successor: ext.GraphID | ext.graphs.Graph) -> ext.GraphID: ...

__init__()

Source code in build-openzl/py/site-packages/openzl/ext/nodes.pyi
def __init__(self) -> None: ...

build_graph(compressor, successors)

Source code in build-openzl/py/site-packages/openzl/ext/nodes.pyi
def build_graph(self, compressor: ext.Compressor, successors: Sequence[ext.GraphID]) -> ext.GraphID: ...

parameterize(compressor)

Source code in build-openzl/py/site-packages/openzl/ext/nodes.pyi
def parameterize(self, compressor: ext.Compressor) -> ext.NodeID: ...

run(edge)

Source code in build-openzl/py/site-packages/openzl/ext/nodes.pyi
def run(self, edge: ext.Edge) -> list[ext.Edge]: ...

run_multi_input(edges)

Source code in build-openzl/py/site-packages/openzl/ext/nodes.pyi
def run_multi_input(self, edges: Sequence[ext.Edge]) -> list[ext.Edge]: ...

openzl.ext.nodes.ConvertStructToSerial

Bases: Node

Convert struct to serial

Inputs: input: Type.Struct

Singleton Outputs: converted: Type.Serial

Source code in build-openzl/py/site-packages/openzl/ext/nodes.pyi
class ConvertStructToSerial(Node):
    """
    Convert struct to serial

    Inputs:
    input: Type.Struct


    Singleton Outputs:
    converted: Type.Serial
    """

    def __init__(self) -> None: ...

    def __call__(self, compressor: ext.Compressor, successor: ext.GraphID | ext.graphs.Graph) -> ext.GraphID: ...

    def run(self, edge: ext.Edge) -> list[ext.Edge]: ...

    def run_multi_input(self, edges: Sequence[ext.Edge]) -> list[ext.Edge]: ...

    def build_graph(self, compressor: ext.Compressor, successors: Sequence[ext.GraphID]) -> ext.GraphID: ...

    def parameterize(self, compressor: ext.Compressor) -> ext.NodeID: ...

    @property
    def base_node(self) -> ext.NodeID: ...

base_node property

__call__(compressor, successor)

Source code in build-openzl/py/site-packages/openzl/ext/nodes.pyi
def __call__(self, compressor: ext.Compressor, successor: ext.GraphID | ext.graphs.Graph) -> ext.GraphID: ...

__init__()

Source code in build-openzl/py/site-packages/openzl/ext/nodes.pyi
def __init__(self) -> None: ...

build_graph(compressor, successors)

Source code in build-openzl/py/site-packages/openzl/ext/nodes.pyi
def build_graph(self, compressor: ext.Compressor, successors: Sequence[ext.GraphID]) -> ext.GraphID: ...

parameterize(compressor)

Source code in build-openzl/py/site-packages/openzl/ext/nodes.pyi
def parameterize(self, compressor: ext.Compressor) -> ext.NodeID: ...

run(edge)

Source code in build-openzl/py/site-packages/openzl/ext/nodes.pyi
def run(self, edge: ext.Edge) -> list[ext.Edge]: ...

run_multi_input(edges)

Source code in build-openzl/py/site-packages/openzl/ext/nodes.pyi
def run_multi_input(self, edges: Sequence[ext.Edge]) -> list[ext.Edge]: ...

openzl.ext.nodes.ConvertSerialToString

Bases: Node

Convert a serial input to a string output by telling OpenZL the string lengths

Inputs: input: Type.Serial

Singleton Outputs: converted: Type.String

Source code in build-openzl/py/site-packages/openzl/ext/nodes.pyi
class ConvertSerialToString(Node):
    """
    Convert a serial input to a string output by telling OpenZL the string lengths

    Inputs:
    input: Type.Serial


    Singleton Outputs:
    converted: Type.String
    """

    def __init__(self, string_lens: Sequence[int]) -> None: ...

    def __call__(self, compressor: ext.Compressor, successor: ext.GraphID | ext.graphs.Graph) -> ext.GraphID: ...

    def run(self, edge: ext.Edge) -> list[ext.Edge]: ...

    def run_multi_input(self, edges: Sequence[ext.Edge]) -> list[ext.Edge]: ...

    def build_graph(self, compressor: ext.Compressor, successors: Sequence[ext.GraphID]) -> ext.GraphID: ...

    def parameterize(self, compressor: ext.Compressor) -> ext.NodeID: ...

    @property
    def base_node(self) -> ext.NodeID: ...

base_node property

__call__(compressor, successor)

Source code in build-openzl/py/site-packages/openzl/ext/nodes.pyi
def __call__(self, compressor: ext.Compressor, successor: ext.GraphID | ext.graphs.Graph) -> ext.GraphID: ...

__init__(string_lens)

Source code in build-openzl/py/site-packages/openzl/ext/nodes.pyi
def __init__(self, string_lens: Sequence[int]) -> None: ...

build_graph(compressor, successors)

Source code in build-openzl/py/site-packages/openzl/ext/nodes.pyi
def build_graph(self, compressor: ext.Compressor, successors: Sequence[ext.GraphID]) -> ext.GraphID: ...

parameterize(compressor)

Source code in build-openzl/py/site-packages/openzl/ext/nodes.pyi
def parameterize(self, compressor: ext.Compressor) -> ext.NodeID: ...

run(edge)

Source code in build-openzl/py/site-packages/openzl/ext/nodes.pyi
def run(self, edge: ext.Edge) -> list[ext.Edge]: ...

run_multi_input(edges)

Source code in build-openzl/py/site-packages/openzl/ext/nodes.pyi
def run_multi_input(self, edges: Sequence[ext.Edge]) -> list[ext.Edge]: ...

openzl.ext.nodes.SeparateStringComponents

Bases: Node

Separate a string input into its content and lengths streams

Inputs: strings: Type.String

Singleton Outputs: string content: Type.Serial 32-bit string lengths: Type.Numeric

Source code in build-openzl/py/site-packages/openzl/ext/nodes.pyi
class SeparateStringComponents(Node):
    """
    Separate a string input into its content and lengths streams

    Inputs:
    strings: Type.String


    Singleton Outputs:
    string content: Type.Serial
    32-bit string lengths: Type.Numeric
    """

    def __init__(self) -> None: ...

    def __call__(self, compressor: ext.Compressor, content: ext.GraphID | ext.graphs.Graph, lengths: ext.GraphID | ext.graphs.Graph) -> ext.GraphID: ...

    def run(self, edge: ext.Edge) -> list[ext.Edge]: ...

    def run_multi_input(self, edges: Sequence[ext.Edge]) -> list[ext.Edge]: ...

    def build_graph(self, compressor: ext.Compressor, successors: Sequence[ext.GraphID]) -> ext.GraphID: ...

    def parameterize(self, compressor: ext.Compressor) -> ext.NodeID: ...

    @property
    def base_node(self) -> ext.NodeID: ...

base_node property

__call__(compressor, content, lengths)

Source code in build-openzl/py/site-packages/openzl/ext/nodes.pyi
def __call__(self, compressor: ext.Compressor, content: ext.GraphID | ext.graphs.Graph, lengths: ext.GraphID | ext.graphs.Graph) -> ext.GraphID: ...

__init__()

Source code in build-openzl/py/site-packages/openzl/ext/nodes.pyi
def __init__(self) -> None: ...

build_graph(compressor, successors)

Source code in build-openzl/py/site-packages/openzl/ext/nodes.pyi
def build_graph(self, compressor: ext.Compressor, successors: Sequence[ext.GraphID]) -> ext.GraphID: ...

parameterize(compressor)

Source code in build-openzl/py/site-packages/openzl/ext/nodes.pyi
def parameterize(self, compressor: ext.Compressor) -> ext.NodeID: ...

run(edge)

Source code in build-openzl/py/site-packages/openzl/ext/nodes.pyi
def run(self, edge: ext.Edge) -> list[ext.Edge]: ...

run_multi_input(edges)

Source code in build-openzl/py/site-packages/openzl/ext/nodes.pyi
def run_multi_input(self, edges: Sequence[ext.Edge]) -> list[ext.Edge]: ...