🔇 🐭 👩🏿‍🤝‍👨🏽 Deserialisasi YAML yang ketat dengan Python dengan pustaka marshmallow 👨‍👩‍👧‍👧 😺 🎣

Tugas awal

Anda perlu membaca konfigurasi non-sepele dari file .yaml.
Struktur config dijelaskan menggunakan kelas data.
Diperlukan pemeriksaan jenis dilakukan selama deserialization dan pengecualian dilemparkan jika datanya tidak valid.

Artinya, sederhananya, Anda memerlukan fungsi dari formulir:

def strict_load_yaml(yaml: str, loaded_type: Type[Any]):
    """
    Here is some magic
    """
    pass

Dan fungsi ini akan digunakan seperti ini:

@dataclass
class MyConfig:
    """
    Here is object tree
    """
    pass

try:
    config = strict_load_yamp(open("config.yaml", "w").read(), MyConfig)
except Exception:
    logging.exception("Config is invalid")

Kelas konfigurasi

File tersebut config.py

terlihat seperti ini:

from dataclasses import dataclass
from enum import Enum
from typing import Optional

class Color(Enum):
    RED = "red"
    GREEN = "green"
    BLUE = "blue"

@dataclass
class BattleStationConfig:
    @dataclass
    class Processor:
        core_count: int
        manufacturer: str

    processor: Processor
    memory_gb: int
    led_color: Optional[Color] = None

Opsi itu tidak berhasil

Masalah aslinya biasa terjadi, bukan? Jadi solusinya harus sepele. Cukup impor pustaka yaml standar dan Anda selesai?

PyYaml load

:

from pprint import pprint

from yaml import load, SafeLoader

yaml = """
processor:
  core_count: 8
  manufacturer: Intel
memory_gb: 8
led_color: red
"""

loaded = load(yaml, Loader=SafeLoader)
pprint(loaded)

{'led_color': 'red',
 'memory_gb': 8,
 'processor': {'core_count': 8, 'manufacturer': 'Intel'}}

Yaml , . , **args

:

parsed_config = BattleStationConfig(**loaded)
pprint(parsed_config)

BattleStationConfig(processor={'core_count': 8, 'manufacturer': 'Intel'}, memory_gb=8, led_color='red')

! ! … -. processor ? .

Python Processor

. stackowerflow.

, yaml-

stackowerflow PyYaml , yaml- . YAMLObject

, config_with_tag.py

:

from dataclasses import dataclass
from enum import Enum
from typing import Optional

from yaml import YAMLObject, SafeLoader

class Color(Enum):
    RED = "red"
    GREEN = "green"
    BLUE = "blue"

@dataclass
class BattleStationConfig(YAMLObject):
    yaml_tag = "!BattleStationConfig"
    yaml_loader = SafeLoader

    @dataclass
    class Processor(YAMLObject):
        yaml_tag = "!Processor"
        yaml_loader = SafeLoader

        core_count: int
        manufacturer: str

    processor: Processor
    memory_gb: int
    led_color: Optional[Color] = None

from pprint import pprint

from yaml import load, SafeLoader

from config_with_tag import BattleStationConfig

yaml = """
--- !BattleStationConfig
processor: !Processor
  core_count: 8
  manufacturer: Intel
memory_gb: 8
led_color: red
"""

a = BattleStationConfig

loaded = load(yaml, Loader=SafeLoader)
pprint(loaded)

BattleStationConfig(processor=BattleStationConfig.Processor(core_count=8, manufacturer='Intel'), memory_gb=8, led_color='red')

. yaml- . , Color

- . YAMLObject

? ? , .

class Color(Enum, YAMLObject):
    RED = "red"
    GREEN = "green"
    BLUE = "blue"

TypeError: metaclass conflict: the metaclass of a derived class must be a (non-strict) subclass of the metaclasses of all its bases

. yaml-, .

marshmallow

stackowerflow marshmallow , JSON-. , , , yaml JSON. class_schema

, -:

from pprint import pprint

from yaml import load, SafeLoader
from marshmallow_dataclass import class_schema

from config import BattleStationConfig

yaml = """
processor:
  core_count: 8
  manufacturer: Intel
memory_gb: 8
led_color: red
"""

loaded = load(yaml, Loader=SafeLoader)
pprint(loaded)

BattleStationConfigSchema = class_schema(BattleStationConfig)

result = BattleStationConfigSchema().load(loaded)
pprint(result)

, , :

marshmallow.exceptions.ValidationError: {'led_color': ['Invalid enum member red']}

, marshmallow enum, . yaml- :

processor:
  core_count: 8
  manufacturer: Intel
memory_gb: 8
led_color: RED

, , :

BattleStationConfig(processor=BattleStationConfig.Processor(core_count=8, manufacturer='Intel'), memory_gb=8, led_color=<Color.RED: 'red'>)

, yaml-. marshmallow :

Setting by_value=True

. This will cause both dumping and loading to use the value of the enum.

, metadata

field

:

@dataclass
class BattleStationConfig:
    led_color: Optional[Color] = field(default=None, metadata={"by_value": True})

, "" , yaml-.

, :

def strict_load_yaml(yaml: str, loaded_type: Type[Any]):
    schema = class_schema(loaded_type)
    return schema().load(load(yaml, Loader=SafeLoader))

Fungsi ini mungkin memerlukan konfigurasi tambahan untuk kelas data, tetapi ini memecahkan masalah asli dan tidak memerlukan tag di yaml.

Catatan singkat tentang ForwardRef

Jika Anda mendefinisikan kelas data dengan ForwardRef (string dengan nama kelas) marshmallow akan bingung dan tidak akan bisa mengurai kelas ini.

Misalnya, konfigurasi seperti itu

from dataclasses import dataclass, field
from enum import Enum
from typing import Optional, ForwardRef

@dataclass
class BattleStationConfig:
    processor: ForwardRef("Processor")
    memory_gb: int
    led_color: Optional["Color"] = field(default=None, metadata={"by_value": True})

    @dataclass
    class Processor:
        core_count: int
        manufacturer: str

class Color(Enum):
    RED = "red"
    GREEN = "green"
    BLUE = "blue"

akan menghasilkan kesalahan

marshmallow.exceptions.RegistryError: Class with name 'Processor' was not found. You may need to import the class.

Dan jika Anda memindahkan kelas Processor

lebih tinggi, marshmallow akan kehilangan kelas Color

dengan kesalahan serupa. Jadi, jika memungkinkan, jangan gunakan ForwardRef di kelas Anda jika Anda ingin menguraikannya dengan marshmallow.

Kode

Semua kode tersedia di repositori GitHub .

Deserialisasi YAML yang ketat dengan Python dengan pustaka marshmallow