textfier.core

The core is the core. Essentially, it is the parent of everything. You should find parent classes defining the basis of our structure. They should provide variables and methods that will help to construct other modules.

Core-based implementations, such as datasets and models.

class textfier.core.Dataset(**kwargs)

Bases: torch.utils.data.Dataset

Dataset implements a default class used to handle customizable datasets.

__getitem__(self, idx: int)

Private method that serves as PyTorch’s iterator.

Parameters

idx – Index of sample.

Returns

Desired keys/values pair.

Return type

(Dict[str, Any])

__init__(self, **kwargs)

Initialization method.

__len__(self)

Private method that serve as PyTorch’s auxiliary.

Returns

Length of the first dataset’s property.

Return type

(int)

class textfier.core.Runner(model: transformers.PreTrainedModel, train_dataset: Optional[textfier.core.dataset.Dataset] = None, eval_dataset: Optional[textfier.core.dataset.Dataset] = None, **kwargs)

Bases: transformers.Trainer

Runner implements a default class used to handle customizable trainers.

__init__(self, model: transformers.PreTrainedModel, train_dataset: Optional[textfier.core.dataset.Dataset] = None, eval_dataset: Optional[textfier.core.dataset.Dataset] = None, **kwargs)

Inialization method.

Parameters

• model – Pre-trained model.

• train_dataset – Training dataset.

• eval_dataset – Evaluation dataset.

class textfier.core.Task(model: str, **kwargs)

Task implements a default class used to handle customizable tasks.

__init__(self, model: str, **kwargs)

Initialization method.

Parameters

model – Identifier of the pre-trained model to be loaded.

abstract _build(self, model: str)

Builds up the pre-trained model according to the desired task.

Parameters

model – Identifier of the pre-trained model to be built.