Skip to content

Concat

Concat Manual

Overview

There is one codec for each type - serial, numeric, struct or string in the family of 'concat' codecs. All inputs must be of the same type and element width for these 'concat' codecs.

The 'concat' codec takes multiple inputs and concatenates them in the order given, returning the result as a single output.

Inputs

The 'concat' codec takes \(N\) inputs, all of the same type and element width. On format version 17 of OpenZL, the maximum number of inputs is 2048, due to engine limitations.

Outputs

There are two outputs produced: - a numeric output, which contains the size of each segment in the input - an input of type \(T\) (either serial, numeric, or struct) and element width \(n\), which contains all segments concatenated in order

Use Cases

Used for combining data with similar formats so they can be compressed in a unified manner. Ideally, entropy statistics of the individual inputs should be similar for compression to be effective.

openzl.ext.nodes.Concat

Bases: Node

Source code in build-openzl/py/site-packages/openzl/ext/nodes.pyi 
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
class Concat(Node):
    def __init__(self, type: ext.Type) -> None: ...

    def __call__(self, compressor: ext.Compressor, lengths: ext.GraphID | ext.graphs.Graph, concatenated: ext.GraphID | ext.graphs.Graph) -> ext.GraphID: ...

    def run(self, edge: ext.Edge) -> list[ext.Edge]: ...

    def run_multi_input(self, edges: Sequence[ext.Edge]) -> list[ext.Edge]: ...

    def build_graph(self, compressor: ext.Compressor, successors: Sequence[ext.GraphID]) -> ext.GraphID: ...

    def parameterize(self, compressor: ext.Compressor) -> ext.NodeID: ...

    @property
    def base_node(self) -> ext.NodeID: ...

base_node property

__call__(compressor, lengths, concatenated)

Source code in build-openzl/py/site-packages/openzl/ext/nodes.pyi 
164
def __call__(self, compressor: ext.Compressor, lengths: ext.GraphID | ext.graphs.Graph, concatenated: ext.GraphID | ext.graphs.Graph) -> ext.GraphID: ...

__init__(type)

Source code in build-openzl/py/site-packages/openzl/ext/nodes.pyi 
162
def __init__(self, type: ext.Type) -> None: ...

build_graph(compressor, successors)

Source code in build-openzl/py/site-packages/openzl/ext/nodes.pyi 
170
def build_graph(self, compressor: ext.Compressor, successors: Sequence[ext.GraphID]) -> ext.GraphID: ...

parameterize(compressor)

Source code in build-openzl/py/site-packages/openzl/ext/nodes.pyi 
172
def parameterize(self, compressor: ext.Compressor) -> ext.NodeID: ...

run(edge)

Source code in build-openzl/py/site-packages/openzl/ext/nodes.pyi 
166
def run(self, edge: ext.Edge) -> list[ext.Edge]: ...

run_multi_input(edges)

Source code in build-openzl/py/site-packages/openzl/ext/nodes.pyi 
168
def run_multi_input(self, edges: Sequence[ext.Edge]) -> list[ext.Edge]: ...