

reconer.dataset.Dataset

reconer.dataset.Dataset is a container to run other reconer operations across your train/dev/test split The reconer.dataset.Dataset.apply function takes any of the other reconer functions and runs them on all the datasets in sequence.

---

API

class reconer.Dataset(train, dev, test=None)

Container for a full dataset with train/dev/test splits. Used to apply core functions to all datasets at once.

Parameters

---

train: (List[Example]), required.

List of Examples for train set

dev: (List[Example]), required.

List of Examples for dev set

test: (List[Example], optional), Defaults to None.

List of Examples for test set

apply(self, func, *args, **kwargs)

Apply an existing function to all datasets

Parameters

---

func: (Callable[[List[Example]], Any]), required.

Function from an existing reconer module that can operate on a List of Examples

Returns

---

(Dict[str, List[Example]]):

Dictionary mapping dataset names to List[Example], same as the internal datasets property

from_disk(path, loader_func=)

Load Dataset from disk given a directory with files named explicitly train.jsonl, dev.jsonl, and test.jsonl

Parameters

---

path: (Path), required.

directory to load from

loader_func: (Callable, optional), Defaults to read_jsonl.

Loader function (TODO: Make this a bit more generic)