miniml.nn.positional

PositionalEmbedding

Bases: MiniMLModel

A MiniML model that adds positional embeddings to input sequences.

max_length property

Maximum length of input sequences.

dim property

Dimensionality of the embedding vectors.

__init__(max_length, dim)

Initialize the PositionalEmbedding model.

Parameters:

Name	Type	Description	Default
max_length	int	The maximum length of input sequences.	required
dim	int	The dimensionality of the embedding vectors.	required

RotaryPositionalEmbedding

Bases: MiniMLModel

A MiniML model that applies rotary positional embeddings to input sequences.

dim property

Dimensionality of the embedding vectors.

__init__(dim, rot_length=10000)

Initialize the RotaryPositionalEmbedding model.

Parameters:

Name	Type	Description	Default
dim	int	The dimensionality of the embedding vectors.	required
rot_length	int	The maximum length of input sequences, after which the embeddings will repeat.	10000