pipeline.blocks package

Submodules

pipeline.blocks.blocks module

class pipeline.blocks.blocks.ConfigBlocks(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)[source]

Bases: Enum

Enum for the config blocks

general = <class 'pipeline.blocks.general_dto.GeneralDTO'>
Parameters:
Return type:

None

data = <class 'pipeline.blocks.data_dto.DataDTO'>
Parameters:

  • csv_file_path (str)

  • validation_split (float)

  • split_seed (int)

Return type:

None

experiment = <class 'pipeline.blocks.experiment_dto.ExperimentDTO'>
Parameters:
Return type:

None

output = <class 'pipeline.blocks.output_dto.OutputDTO'>
Parameters:

  • results_dir (str)

  • move_results_to (str | None)

Return type:

None

pipeline.blocks.data_dto module

class pipeline.blocks.data_dto.DataDTO(*, csv_file_path, validation_split=0.2, split_seed=42)[source]

Bases: BaseModel

Parameters:

  • csv_file_path (str)

  • validation_split (float)

  • split_seed (int)

csv_file_path: str
validation_split: float
split_seed: int
classmethod validation_split_validator(v)[source]
property images_directory: str
model_computed_fields: ClassVar[dict[str, ComputedFieldInfo]] = {'images_directory': ComputedFieldInfo(wrapped_property=<property object>, return_type=<class 'str'>, alias=None, alias_priority=None, title=None, description=None, deprecated=None, examples=None, json_schema_extra=None, repr=True)}

A dictionary of computed field names and their corresponding ComputedFieldInfo objects.

model_config: ClassVar[ConfigDict] = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[dict[str, FieldInfo]] = {'csv_file_path': FieldInfo(annotation=str, required=True), 'split_seed': FieldInfo(annotation=int, required=False, default=42), 'validation_split': FieldInfo(annotation=float, required=False, default=0.2)}

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].

This replaces Model.__fields__ from Pydantic V1.

pipeline.blocks.experiment_dto module

class pipeline.blocks.experiment_dto.LoopObjectDTO(*, name, repeat=True)[source]

Bases: BaseModel

Parameters:
model_config: ClassVar[ConfigDict] = {'protected_namespaces': ()}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

name: str
repeat: bool
classmethod name_validator(v)[source]
model_computed_fields: ClassVar[dict[str, ComputedFieldInfo]] = {}

A dictionary of computed field names and their corresponding ComputedFieldInfo objects.

model_fields: ClassVar[dict[str, FieldInfo]] = {'name': FieldInfo(annotation=str, required=True), 'repeat': FieldInfo(annotation=bool, required=False, default=True)}

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].

This replaces Model.__fields__ from Pydantic V1.

class pipeline.blocks.experiment_dto.TrainGeneratorDTO(*, name, repeat=True, batch_size, lr, save_and_sample_every, num_steps, results_dir, gradient_accumulate_every=4, experiment_id=None, copy_results_to=None, start_from_checkpoint=None, dataset_split_type='train', diagnosis)[source]

Bases: LoopObjectDTO

Parameters:

  • name (str)

  • repeat (bool)

  • batch_size (int)

  • lr (float)

  • save_and_sample_every (int)

  • num_steps (int)

  • results_dir (str)

  • gradient_accumulate_every (int)

  • experiment_id (str | None)

  • copy_results_to (str | None)

  • start_from_checkpoint (str | None)

  • dataset_split_type (Literal['train', 'val', 'test'])

  • diagnosis (Literal['precancerous', 'fluid', 'benign', 'reference'])

batch_size: int
lr: float
save_and_sample_every: int
num_steps: int
results_dir: str
gradient_accumulate_every: int
experiment_id: str | None
copy_results_to: str | None
start_from_checkpoint: str | None
dataset_split_type: Literal['train', 'val', 'test']
diagnosis: Literal['precancerous', 'fluid', 'benign', 'reference']
classmethod name_validator(v)[source]
model_computed_fields: ClassVar[dict[str, ComputedFieldInfo]] = {}

A dictionary of computed field names and their corresponding ComputedFieldInfo objects.

model_config: ClassVar[ConfigDict] = {'protected_namespaces': ()}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[dict[str, FieldInfo]] = {'batch_size': FieldInfo(annotation=int, required=True), 'copy_results_to': FieldInfo(annotation=Union[str, NoneType], required=False, default=None), 'dataset_split_type': FieldInfo(annotation=Literal['train', 'val', 'test'], required=False, default='train'), 'diagnosis': FieldInfo(annotation=Literal['precancerous', 'fluid', 'benign', 'reference'], required=True), 'experiment_id': FieldInfo(annotation=Union[str, NoneType], required=False, default=None), 'gradient_accumulate_every': FieldInfo(annotation=int, required=False, default=4), 'lr': FieldInfo(annotation=float, required=True), 'name': FieldInfo(annotation=str, required=True), 'num_steps': FieldInfo(annotation=int, required=True), 'repeat': FieldInfo(annotation=bool, required=False, default=True), 'results_dir': FieldInfo(annotation=str, required=True), 'save_and_sample_every': FieldInfo(annotation=int, required=True), 'start_from_checkpoint': FieldInfo(annotation=Union[str, NoneType], required=False, default=None)}

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].

This replaces Model.__fields__ from Pydantic V1.

class pipeline.blocks.experiment_dto.GenerateSamplesDTO(*, name, repeat=True, num_samples, batch_size, wandb=True, model_version, base_on, results_dir, copy_results_to=None, relative_dataset_results_dir='dataset')[source]

Bases: LoopObjectDTO

Parameters:

  • name (str)

  • repeat (bool)

  • num_samples (int)

  • batch_size (int)

  • wandb (bool)

  • model_version (str)

  • base_on (str)

  • results_dir (str)

  • copy_results_to (str | None)

  • relative_dataset_results_dir (str)

num_samples: int
batch_size: int
wandb: bool
model_version: str
base_on: str
results_dir: str
copy_results_to: str | None
relative_dataset_results_dir: str
property checkpoint_path: str
property generete_samples_dir: str
classmethod name_validator(v)[source]
model_computed_fields: ClassVar[dict[str, ComputedFieldInfo]] = {'checkpoint_path': ComputedFieldInfo(wrapped_property=<property object>, return_type=<class 'str'>, alias=None, alias_priority=None, title=None, description=None, deprecated=None, examples=None, json_schema_extra=None, repr=True), 'generete_samples_dir': ComputedFieldInfo(wrapped_property=<property object>, return_type=<class 'str'>, alias=None, alias_priority=None, title=None, description=None, deprecated=None, examples=None, json_schema_extra=None, repr=True)}

A dictionary of computed field names and their corresponding ComputedFieldInfo objects.

model_config: ClassVar[ConfigDict] = {'protected_namespaces': ()}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[dict[str, FieldInfo]] = {'base_on': FieldInfo(annotation=str, required=True), 'batch_size': FieldInfo(annotation=int, required=True), 'copy_results_to': FieldInfo(annotation=Union[str, NoneType], required=False, default=None), 'model_version': FieldInfo(annotation=str, required=True), 'name': FieldInfo(annotation=str, required=True), 'num_samples': FieldInfo(annotation=int, required=True), 'relative_dataset_results_dir': FieldInfo(annotation=str, required=False, default='dataset'), 'repeat': FieldInfo(annotation=bool, required=False, default=True), 'results_dir': FieldInfo(annotation=str, required=True), 'wandb': FieldInfo(annotation=bool, required=False, default=True)}

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].

This replaces Model.__fields__ from Pydantic V1.

class pipeline.blocks.experiment_dto.FooDTO(*, name, repeat=True, foo='foo')[source]

Bases: LoopObjectDTO

Parameters:
foo: str
classmethod name_validator(v)[source]
model_computed_fields: ClassVar[dict[str, ComputedFieldInfo]] = {}

A dictionary of computed field names and their corresponding ComputedFieldInfo objects.

model_config: ClassVar[ConfigDict] = {'protected_namespaces': ()}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[dict[str, FieldInfo]] = {'foo': FieldInfo(annotation=str, required=False, default='foo'), 'name': FieldInfo(annotation=str, required=True), 'repeat': FieldInfo(annotation=bool, required=False, default=True)}

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].

This replaces Model.__fields__ from Pydantic V1.

class pipeline.blocks.experiment_dto.ClassificationDTO(*, epochs, lr, loss_multiply=1.0, ratio=[0.8, 0.2], class_names=['fluid', 'benign', 'precancerous', 'reference'], train_data_type='synthetic', train_dataset_dir, val_dataset_dir=None, test_dataset_dir, loss_fn='cross_entropy', num_workers=4, batch_size=32, log_every_n_steps=10, logger_tags=None, logger_experiment_name=None, offline=False, results_dir)[source]

Bases: BaseModel

Parameters:

  • epochs (int)

  • lr (float)

  • loss_multiply (float)

  • ratio (list[float])

  • class_names (list[str])

  • train_data_type (Literal['real', 'synthetic'])

  • train_dataset_dir (str)

  • val_dataset_dir (str | None)

  • test_dataset_dir (str)

  • loss_fn (str)

  • num_workers (int)

  • batch_size (int)

  • log_every_n_steps (int)

  • logger_tags (list[str] | None)

  • logger_experiment_name (str | None)

  • offline (bool)

  • results_dir (str)

epochs: int
lr: float
loss_multiply: float
ratio: list[float]
class_names: list[str]
train_data_type: Literal['real', 'synthetic']
train_dataset_dir: str
val_dataset_dir: str | None
test_dataset_dir: str
loss_fn: str
num_workers: int
batch_size: int
log_every_n_steps: int
logger_tags: list[str] | None
logger_experiment_name: str | None
offline: bool
results_dir: str
property num_classes: int
model_computed_fields: ClassVar[dict[str, ComputedFieldInfo]] = {'num_classes': ComputedFieldInfo(wrapped_property=<property object>, return_type=<class 'int'>, alias=None, alias_priority=None, title=None, description=None, deprecated=None, examples=None, json_schema_extra=None, repr=True)}

A dictionary of computed field names and their corresponding ComputedFieldInfo objects.

model_config: ClassVar[ConfigDict] = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[dict[str, FieldInfo]] = {'batch_size': FieldInfo(annotation=int, required=False, default=32), 'class_names': FieldInfo(annotation=list[str], required=False, default=['fluid', 'benign', 'precancerous', 'reference']), 'epochs': FieldInfo(annotation=int, required=True), 'log_every_n_steps': FieldInfo(annotation=int, required=False, default=10), 'logger_experiment_name': FieldInfo(annotation=Union[str, NoneType], required=False, default=None), 'logger_tags': FieldInfo(annotation=Union[list[str], NoneType], required=False, default=None), 'loss_fn': FieldInfo(annotation=str, required=False, default='cross_entropy'), 'loss_multiply': FieldInfo(annotation=float, required=False, default=1.0), 'lr': FieldInfo(annotation=float, required=True), 'num_workers': FieldInfo(annotation=int, required=False, default=4), 'offline': FieldInfo(annotation=bool, required=False, default=False), 'ratio': FieldInfo(annotation=list[float], required=False, default=[0.8, 0.2]), 'results_dir': FieldInfo(annotation=str, required=True), 'test_dataset_dir': FieldInfo(annotation=str, required=True), 'train_data_type': FieldInfo(annotation=Literal['real', 'synthetic'], required=False, default='synthetic'), 'train_dataset_dir': FieldInfo(annotation=str, required=True), 'val_dataset_dir': FieldInfo(annotation=Union[str, NoneType], required=False, default=None)}

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].

This replaces Model.__fields__ from Pydantic V1.

class pipeline.blocks.experiment_dto.ValidateDTO(*, name, repeat=True, results_dir, copy_results_to=None, classification=None)[source]

Bases: LoopObjectDTO

Parameters:
results_dir: str
copy_results_to: str | None
classification: ClassificationDTO | None
classmethod name_validator(v)[source]
classmethod classification_validator(v, values)[source]
model_computed_fields: ClassVar[dict[str, ComputedFieldInfo]] = {}

A dictionary of computed field names and their corresponding ComputedFieldInfo objects.

model_config: ClassVar[ConfigDict] = {'protected_namespaces': ()}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[dict[str, FieldInfo]] = {'classification': FieldInfo(annotation=Union[ClassificationDTO, NoneType], required=False, default=None), 'copy_results_to': FieldInfo(annotation=Union[str, NoneType], required=False, default=None), 'name': FieldInfo(annotation=str, required=True), 'repeat': FieldInfo(annotation=bool, required=False, default=True), 'results_dir': FieldInfo(annotation=str, required=True)}

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].

This replaces Model.__fields__ from Pydantic V1.

class pipeline.blocks.experiment_dto.ExperimentDTO(*, total_steps, image_size, models, results_dir, copy_results_to=None, loop)[source]

Bases: BaseModel

Parameters:
total_steps: int
image_size: list[int]
models: ModelsDTO
results_dir: str
copy_results_to: str | None
loop: list[TrainGeneratorDTO | GenerateSamplesDTO | ValidateDTO | FooDTO]
classmethod loop_validator(v, values)[source]
model_computed_fields: ClassVar[dict[str, ComputedFieldInfo]] = {}

A dictionary of computed field names and their corresponding ComputedFieldInfo objects.

model_config: ClassVar[ConfigDict] = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[dict[str, FieldInfo]] = {'copy_results_to': FieldInfo(annotation=Union[str, NoneType], required=False, default=None), 'image_size': FieldInfo(annotation=list[int], required=True), 'loop': FieldInfo(annotation=list[Union[TrainGeneratorDTO, GenerateSamplesDTO, ValidateDTO, FooDTO]], required=True), 'models': FieldInfo(annotation=ModelsDTO, required=True), 'results_dir': FieldInfo(annotation=str, required=True), 'total_steps': FieldInfo(annotation=int, required=True)}

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].

This replaces Model.__fields__ from Pydantic V1.

pipeline.blocks.general_dto module

class pipeline.blocks.general_dto.GeneralDTO(*, total_steps=0, image_size, experiment_id=None, models)[source]

Bases: BaseModel

Parameters:
total_steps: int
image_size: list[int]
experiment_id: str | None
models: ModelsDTO
model_computed_fields: ClassVar[dict[str, ComputedFieldInfo]] = {}

A dictionary of computed field names and their corresponding ComputedFieldInfo objects.

model_config: ClassVar[ConfigDict] = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[dict[str, FieldInfo]] = {'experiment_id': FieldInfo(annotation=Union[str, NoneType], required=False, default=None), 'image_size': FieldInfo(annotation=list[int], required=True), 'models': FieldInfo(annotation=ModelsDTO, required=True), 'total_steps': FieldInfo(annotation=int, required=False, default=0)}

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].

This replaces Model.__fields__ from Pydantic V1.

pipeline.blocks.models_dto module

class pipeline.blocks.models_dto.ModelsDTO(*, unet=None, diffusion=None, classifier=None)[source]

Bases: BaseModel

Parameters:
unet: dict[str, Any] | None
diffusion: dict[str, Any] | None
classifier: dict[str, Any] | None
model_computed_fields: ClassVar[dict[str, ComputedFieldInfo]] = {}

A dictionary of computed field names and their corresponding ComputedFieldInfo objects.

model_config: ClassVar[ConfigDict] = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[dict[str, FieldInfo]] = {'classifier': FieldInfo(annotation=Union[dict[str, Any], NoneType], required=False, default=None), 'diffusion': FieldInfo(annotation=Union[dict[str, Any], NoneType], required=False, default=None), 'unet': FieldInfo(annotation=Union[dict[str, Any], NoneType], required=False, default=None)}

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].

This replaces Model.__fields__ from Pydantic V1.

pipeline.blocks.output_dto module

class pipeline.blocks.output_dto.OutputDTO(*, results_dir='.results', move_results_to=None)[source]

Bases: BaseModel

Parameters:

  • results_dir (str)

  • move_results_to (str | None)

results_dir: str
move_results_to: str | None
model_computed_fields: ClassVar[dict[str, ComputedFieldInfo]] = {}

A dictionary of computed field names and their corresponding ComputedFieldInfo objects.

model_config: ClassVar[ConfigDict] = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[dict[str, FieldInfo]] = {'move_results_to': FieldInfo(annotation=Union[str, NoneType], required=False, default=None), 'results_dir': FieldInfo(annotation=str, required=False, default='.results')}

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].

This replaces Model.__fields__ from Pydantic V1.

pipeline.blocks.testing_dto module

Module contents

class pipeline.blocks.ConfigBlocks(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)[source]

Bases: Enum

Enum for the config blocks

general = <class 'pipeline.blocks.general_dto.GeneralDTO'>
Parameters:
Return type:

None

data = <class 'pipeline.blocks.data_dto.DataDTO'>
Parameters:

  • csv_file_path (str)

  • validation_split (float)

  • split_seed (int)

Return type:

None

experiment = <class 'pipeline.blocks.experiment_dto.ExperimentDTO'>
Parameters:
Return type:

None

output = <class 'pipeline.blocks.output_dto.OutputDTO'>
Parameters:

  • results_dir (str)

  • move_results_to (str | None)

Return type:

None

class pipeline.blocks.FooDTO(*, name, repeat=True, foo='foo')[source]

Bases: LoopObjectDTO

Parameters:
foo: str
classmethod name_validator(v)[source]
model_computed_fields: ClassVar[dict[str, ComputedFieldInfo]] = {}

A dictionary of computed field names and their corresponding ComputedFieldInfo objects.

model_config: ClassVar[ConfigDict] = {'protected_namespaces': ()}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[dict[str, FieldInfo]] = {'foo': FieldInfo(annotation=str, required=False, default='foo'), 'name': FieldInfo(annotation=str, required=True), 'repeat': FieldInfo(annotation=bool, required=False, default=True)}

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].

This replaces Model.__fields__ from Pydantic V1.

name: str
repeat: bool
class pipeline.blocks.GenerateSamplesDTO(*, name, repeat=True, num_samples, batch_size, wandb=True, model_version, base_on, results_dir, copy_results_to=None, relative_dataset_results_dir='dataset')[source]

Bases: LoopObjectDTO

Parameters:

  • name (str)

  • repeat (bool)

  • num_samples (int)

  • batch_size (int)

  • wandb (bool)

  • model_version (str)

  • base_on (str)

  • results_dir (str)

  • copy_results_to (str | None)

  • relative_dataset_results_dir (str)

num_samples: int
batch_size: int
wandb: bool
model_version: str
base_on: str
results_dir: str
copy_results_to: str | None
relative_dataset_results_dir: str
property checkpoint_path: str
property generete_samples_dir: str
classmethod name_validator(v)[source]
model_computed_fields: ClassVar[dict[str, ComputedFieldInfo]] = {'checkpoint_path': ComputedFieldInfo(wrapped_property=<property object>, return_type=<class 'str'>, alias=None, alias_priority=None, title=None, description=None, deprecated=None, examples=None, json_schema_extra=None, repr=True), 'generete_samples_dir': ComputedFieldInfo(wrapped_property=<property object>, return_type=<class 'str'>, alias=None, alias_priority=None, title=None, description=None, deprecated=None, examples=None, json_schema_extra=None, repr=True)}

A dictionary of computed field names and their corresponding ComputedFieldInfo objects.

model_config: ClassVar[ConfigDict] = {'protected_namespaces': ()}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[dict[str, FieldInfo]] = {'base_on': FieldInfo(annotation=str, required=True), 'batch_size': FieldInfo(annotation=int, required=True), 'copy_results_to': FieldInfo(annotation=Union[str, NoneType], required=False, default=None), 'model_version': FieldInfo(annotation=str, required=True), 'name': FieldInfo(annotation=str, required=True), 'num_samples': FieldInfo(annotation=int, required=True), 'relative_dataset_results_dir': FieldInfo(annotation=str, required=False, default='dataset'), 'repeat': FieldInfo(annotation=bool, required=False, default=True), 'results_dir': FieldInfo(annotation=str, required=True), 'wandb': FieldInfo(annotation=bool, required=False, default=True)}

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].

This replaces Model.__fields__ from Pydantic V1.

name: str
repeat: bool
class pipeline.blocks.TrainGeneratorDTO(*, name, repeat=True, batch_size, lr, save_and_sample_every, num_steps, results_dir, gradient_accumulate_every=4, experiment_id=None, copy_results_to=None, start_from_checkpoint=None, dataset_split_type='train', diagnosis)[source]

Bases: LoopObjectDTO

Parameters:

  • name (str)

  • repeat (bool)

  • batch_size (int)

  • lr (float)

  • save_and_sample_every (int)

  • num_steps (int)

  • results_dir (str)

  • gradient_accumulate_every (int)

  • experiment_id (str | None)

  • copy_results_to (str | None)

  • start_from_checkpoint (str | None)

  • dataset_split_type (Literal['train', 'val', 'test'])

  • diagnosis (Literal['precancerous', 'fluid', 'benign', 'reference'])

batch_size: int
lr: float
save_and_sample_every: int
num_steps: int
results_dir: str
gradient_accumulate_every: int
experiment_id: str | None
copy_results_to: str | None
start_from_checkpoint: str | None
dataset_split_type: Literal['train', 'val', 'test']
diagnosis: Literal['precancerous', 'fluid', 'benign', 'reference']
classmethod name_validator(v)[source]
model_computed_fields: ClassVar[dict[str, ComputedFieldInfo]] = {}

A dictionary of computed field names and their corresponding ComputedFieldInfo objects.

model_config: ClassVar[ConfigDict] = {'protected_namespaces': ()}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[dict[str, FieldInfo]] = {'batch_size': FieldInfo(annotation=int, required=True), 'copy_results_to': FieldInfo(annotation=Union[str, NoneType], required=False, default=None), 'dataset_split_type': FieldInfo(annotation=Literal['train', 'val', 'test'], required=False, default='train'), 'diagnosis': FieldInfo(annotation=Literal['precancerous', 'fluid', 'benign', 'reference'], required=True), 'experiment_id': FieldInfo(annotation=Union[str, NoneType], required=False, default=None), 'gradient_accumulate_every': FieldInfo(annotation=int, required=False, default=4), 'lr': FieldInfo(annotation=float, required=True), 'name': FieldInfo(annotation=str, required=True), 'num_steps': FieldInfo(annotation=int, required=True), 'repeat': FieldInfo(annotation=bool, required=False, default=True), 'results_dir': FieldInfo(annotation=str, required=True), 'save_and_sample_every': FieldInfo(annotation=int, required=True), 'start_from_checkpoint': FieldInfo(annotation=Union[str, NoneType], required=False, default=None)}

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].

This replaces Model.__fields__ from Pydantic V1.

name: str
repeat: bool
class pipeline.blocks.ValidateDTO(*, name, repeat=True, results_dir, copy_results_to=None, classification=None)[source]

Bases: LoopObjectDTO

Parameters:
results_dir: str
copy_results_to: str | None
classification: ClassificationDTO | None
classmethod name_validator(v)[source]
classmethod classification_validator(v, values)[source]
model_computed_fields: ClassVar[dict[str, ComputedFieldInfo]] = {}

A dictionary of computed field names and their corresponding ComputedFieldInfo objects.

model_config: ClassVar[ConfigDict] = {'protected_namespaces': ()}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[dict[str, FieldInfo]] = {'classification': FieldInfo(annotation=Union[ClassificationDTO, NoneType], required=False, default=None), 'copy_results_to': FieldInfo(annotation=Union[str, NoneType], required=False, default=None), 'name': FieldInfo(annotation=str, required=True), 'repeat': FieldInfo(annotation=bool, required=False, default=True), 'results_dir': FieldInfo(annotation=str, required=True)}

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].

This replaces Model.__fields__ from Pydantic V1.

name: str
repeat: bool
class pipeline.blocks.DataDTO(*, csv_file_path, validation_split=0.2, split_seed=42)[source]

Bases: BaseModel

Parameters:

  • csv_file_path (str)

  • validation_split (float)

  • split_seed (int)

csv_file_path: str
validation_split: float
split_seed: int
classmethod validation_split_validator(v)[source]
property images_directory: str
model_computed_fields: ClassVar[dict[str, ComputedFieldInfo]] = {'images_directory': ComputedFieldInfo(wrapped_property=<property object>, return_type=<class 'str'>, alias=None, alias_priority=None, title=None, description=None, deprecated=None, examples=None, json_schema_extra=None, repr=True)}

A dictionary of computed field names and their corresponding ComputedFieldInfo objects.

model_config: ClassVar[ConfigDict] = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[dict[str, FieldInfo]] = {'csv_file_path': FieldInfo(annotation=str, required=True), 'split_seed': FieldInfo(annotation=int, required=False, default=42), 'validation_split': FieldInfo(annotation=float, required=False, default=0.2)}

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].

This replaces Model.__fields__ from Pydantic V1.

class pipeline.blocks.ClassificationDTO(*, epochs, lr, loss_multiply=1.0, ratio=[0.8, 0.2], class_names=['fluid', 'benign', 'precancerous', 'reference'], train_data_type='synthetic', train_dataset_dir, val_dataset_dir=None, test_dataset_dir, loss_fn='cross_entropy', num_workers=4, batch_size=32, log_every_n_steps=10, logger_tags=None, logger_experiment_name=None, offline=False, results_dir)[source]

Bases: BaseModel

Parameters:

  • epochs (int)

  • lr (float)

  • loss_multiply (float)

  • ratio (list[float])

  • class_names (list[str])

  • train_data_type (Literal['real', 'synthetic'])

  • train_dataset_dir (str)

  • val_dataset_dir (str | None)

  • test_dataset_dir (str)

  • loss_fn (str)

  • num_workers (int)

  • batch_size (int)

  • log_every_n_steps (int)

  • logger_tags (list[str] | None)

  • logger_experiment_name (str | None)

  • offline (bool)

  • results_dir (str)

epochs: int
lr: float
loss_multiply: float
ratio: list[float]
class_names: list[str]
train_data_type: Literal['real', 'synthetic']
train_dataset_dir: str
val_dataset_dir: str | None
test_dataset_dir: str
loss_fn: str
num_workers: int
batch_size: int
log_every_n_steps: int
logger_tags: list[str] | None
logger_experiment_name: str | None
offline: bool
results_dir: str
property num_classes: int
model_computed_fields: ClassVar[dict[str, ComputedFieldInfo]] = {'num_classes': ComputedFieldInfo(wrapped_property=<property object>, return_type=<class 'int'>, alias=None, alias_priority=None, title=None, description=None, deprecated=None, examples=None, json_schema_extra=None, repr=True)}

A dictionary of computed field names and their corresponding ComputedFieldInfo objects.

model_config: ClassVar[ConfigDict] = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[dict[str, FieldInfo]] = {'batch_size': FieldInfo(annotation=int, required=False, default=32), 'class_names': FieldInfo(annotation=list[str], required=False, default=['fluid', 'benign', 'precancerous', 'reference']), 'epochs': FieldInfo(annotation=int, required=True), 'log_every_n_steps': FieldInfo(annotation=int, required=False, default=10), 'logger_experiment_name': FieldInfo(annotation=Union[str, NoneType], required=False, default=None), 'logger_tags': FieldInfo(annotation=Union[list[str], NoneType], required=False, default=None), 'loss_fn': FieldInfo(annotation=str, required=False, default='cross_entropy'), 'loss_multiply': FieldInfo(annotation=float, required=False, default=1.0), 'lr': FieldInfo(annotation=float, required=True), 'num_workers': FieldInfo(annotation=int, required=False, default=4), 'offline': FieldInfo(annotation=bool, required=False, default=False), 'ratio': FieldInfo(annotation=list[float], required=False, default=[0.8, 0.2]), 'results_dir': FieldInfo(annotation=str, required=True), 'test_dataset_dir': FieldInfo(annotation=str, required=True), 'train_data_type': FieldInfo(annotation=Literal['real', 'synthetic'], required=False, default='synthetic'), 'train_dataset_dir': FieldInfo(annotation=str, required=True), 'val_dataset_dir': FieldInfo(annotation=Union[str, NoneType], required=False, default=None)}

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].

This replaces Model.__fields__ from Pydantic V1.