Extending vortex using plugin packages#

The vortex plugin system makes it easy for users to define and redistribute their own resources and task classes.

The vortex and vortex.nwp packages ship with a number of classes that model data, programs and computational tasks that typically make up most Numerical Weather Prediction pipelines. We expect use-cases, however, for which none of the provided classes apply. These could arise from novel research or using vortex within a domain for which it was not originally designed for.

You can implement the appropriate classes fitting your use case by extending one or several classes present in the vortex or vortex.nwp packages. In order to make these classes available to functions like vortex.input(), vortex.executable() or vortex.task(), the corresponding Python modules must be installed as part of a package that declares the vtx entry point in its metadata.

The vortex.loaded_plugins() can be used to list currently loaded plugin packages.

Example#

Let’s say we wish to define the following ressource class and make it available alongside any other vortex ressources. By make it available, we mean that, given the appropriate collection of arguments, functions of the top-level vortex module such as input or output will be able to instantiate the class.

# new.py

from vortex.data.flow import FlowResource

class NewResource(FlowResource):
    _footprint = [
        {
            "kind": {
                "values": ["observations"]
            },
            "nativefmt": {
                "values": ["json"]
            },
            "model": {
                "values": ["arpege"]
            },
        }
    ]

    @property
    def realkind(self):
        return "my-new-ressource"

Assuming the new.py module is installed, importing it will make the NewResource class available:

>>> from new import NewResource
>>> rh = vortex.input(
...    kind="newressource", model="arpege", cutoff="assim"
...    nativefmt="json", experiment="dummy-xp", local="local-filename"
... )[0]
>>> type(rh.resource)
<class 'new.NewResource'>

Having to manually import modules so that they are available through the top level vortex module functions is error-prone. It can also be awkward to import one or more modules for the sole purpose of making a few classes available, with the modules object themselves not being used in the script. For these reasons, we recommend that you group your modules into an plugin package that will be automatically discovered when importing vortex.

Creating a vortex plugin package#

If you group your module(s) into a Python package that declare the vtx entry point as part of its metadata, a import vortex statement will discover and load the modules it ships.

See also

This guide assumes that you are familiar with writing Python package metadata. If this is not the case, see the Configuring metadata section in the Python Packaging User Guide chapter on Packaging Python Projects.

Continuing on the previous example, here is how it works. We now assume that the new.py module is part of a package with the following directory structure:

new-resource-package/
    pyproject.toml
    src/
        vortex_newresource/
             new.py
             __init__.py

The package’s __init__.py module is responsible for making new available under the vortex_newresource namespace:

# src/newresource/__init__.py

# Use a redundant import alias to instruct linters (e.g. Ruff) that this
# is just a re-export of "new" as part of the package's public interface.
from . import new as new

In other words, importing the vortex_newresource package (import vortex_newresource) will make available the new module as newresource.new.

Next, the package metadata must declare the vtx entry point:

# pyproject.toml

[project.entry-points.vtx]
newresource = "vortex_newresource"

Extending vortex using plugin packages

Contents

Extending vortex using plugin packages#

Example#

Creating a vortex plugin package#