使用自定义标记

以下是一些使用 如何用属性标记测试函数 机制的示例。

标记测试函数并选择运行它们

你可以像这样用自定义元数据“标记”一个测试函数

# content of test_server.py

import pytest


@pytest.mark.webtest
def test_send_http():
    pass  # perform some webtest test for your app


@pytest.mark.device(serial="123")
def test_something_quick():
    pass


@pytest.mark.device(serial="abc")
def test_another():
    pass


class TestClass:
    def test_method(self):
        pass

然后，你可以限制测试运行，仅执行标记为 webtest 的测试

$ pytest -v -m webtest
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-9.x.y, pluggy-1.x.y -- $PYTHON_PREFIX/bin/python
cachedir: .pytest_cache
rootdir: /home/sweet/project
collecting ... collected 4 items / 3 deselected / 1 selected

test_server.py::test_send_http PASSED                                [100%]

===================== 1 passed, 3 deselected in 0.12s ======================

或者反过来，运行除 webtest 之外的所有测试

$ pytest -v -m "not webtest"
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-9.x.y, pluggy-1.x.y -- $PYTHON_PREFIX/bin/python
cachedir: .pytest_cache
rootdir: /home/sweet/project
collecting ... collected 4 items / 1 deselected / 3 selected

test_server.py::test_something_quick PASSED                          [ 33%]
test_server.py::test_another PASSED                                  [ 66%]
test_server.py::TestClass::test_method PASSED                        [100%]

===================== 3 passed, 1 deselected in 0.12s ======================

此外，你可以限制测试运行，仅执行匹配一个或多个标记关键字参数的测试，例如仅运行标记为 device 且具有特定 serial="123" 的测试

$ pytest -v -m "device(serial='123')"
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-9.x.y, pluggy-1.x.y -- $PYTHON_PREFIX/bin/python
cachedir: .pytest_cache
rootdir: /home/sweet/project
collecting ... collected 4 items / 3 deselected / 1 selected

test_server.py::test_something_quick PASSED                          [100%]

===================== 1 passed, 3 deselected in 0.12s ======================

注意

标记表达式中仅支持关键字参数匹配。

注意

标记表达式中仅支持 int, (未转义的) str, bool 和 None 值。

基于节点 ID 选择测试

你可以提供一个或多个 节点 ID 作为位置参数，以仅选择指定的测试。这使得基于模块、类、方法或函数名称来选择测试变得很容易

$ pytest -v test_server.py::TestClass::test_method
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-9.x.y, pluggy-1.x.y -- $PYTHON_PREFIX/bin/python
cachedir: .pytest_cache
rootdir: /home/sweet/project
collecting ... collected 1 item

test_server.py::TestClass::test_method PASSED                        [100%]

============================ 1 passed in 0.12s =============================

你也可以按类选择

$ pytest -v test_server.py::TestClass
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-9.x.y, pluggy-1.x.y -- $PYTHON_PREFIX/bin/python
cachedir: .pytest_cache
rootdir: /home/sweet/project
collecting ... collected 1 item

test_server.py::TestClass::test_method PASSED                        [100%]

============================ 1 passed in 0.12s =============================

或者选择多个节点

$ pytest -v test_server.py::TestClass test_server.py::test_send_http
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-9.x.y, pluggy-1.x.y -- $PYTHON_PREFIX/bin/python
cachedir: .pytest_cache
rootdir: /home/sweet/project
collecting ... collected 2 items

test_server.py::TestClass::test_method PASSED                        [ 50%]
test_server.py::test_send_http PASSED                                [100%]

============================ 2 passed in 0.12s =============================

注意

节点 ID 的格式为 module.py::class::method 或 module.py::function。节点 ID 控制收集哪些测试，因此 module.py::class 将选择该类上的所有测试方法。参数化 fixture 或测试的每个参数也会创建节点，因此选择参数化测试时必须包含参数值，例如 module.py::function[param]。

使用 -rf 选项运行 pytest 时，失败测试的节点 ID 会显示在测试摘要信息中。你也可以从 pytest --collect-only 的输出中构建节点 ID。

使用 -k expr 基于名称选择测试

在 2.0/2.3.4 版本中添加。

你可以使用 -k 命令行选项指定一个表达式，该表达式对测试名称进行子字符串匹配，而不是像 -m 那样对标记进行精确匹配。这使得基于名称选择测试变得很容易

在 5.4 版本中更改。

表达式匹配现在不区分大小写。

$ pytest -v -k http  # running with the above defined example module
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-9.x.y, pluggy-1.x.y -- $PYTHON_PREFIX/bin/python
cachedir: .pytest_cache
rootdir: /home/sweet/project
collecting ... collected 4 items / 3 deselected / 1 selected

test_server.py::test_send_http PASSED                                [100%]

===================== 1 passed, 3 deselected in 0.12s ======================

你也可以运行除了匹配关键字之外的所有测试

$ pytest -k "not send_http" -v
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-9.x.y, pluggy-1.x.y -- $PYTHON_PREFIX/bin/python
cachedir: .pytest_cache
rootdir: /home/sweet/project
collecting ... collected 4 items / 1 deselected / 3 selected

test_server.py::test_something_quick PASSED                          [ 33%]
test_server.py::test_another PASSED                                  [ 66%]
test_server.py::TestClass::test_method PASSED                        [100%]

===================== 3 passed, 1 deselected in 0.12s ======================

或者选择“http”和“quick”测试

$ pytest -k "http or quick" -v
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-9.x.y, pluggy-1.x.y -- $PYTHON_PREFIX/bin/python
cachedir: .pytest_cache
rootdir: /home/sweet/project
collecting ... collected 4 items / 2 deselected / 2 selected

test_server.py::test_send_http PASSED                                [ 50%]
test_server.py::test_something_quick PASSED                          [100%]

===================== 2 passed, 2 deselected in 0.12s ======================

你可以使用 and, or, not 和括号。

除了测试名称外，-k 还会匹配测试父级的名称（通常是其所在文件和类的名称）、测试函数上设置的属性、应用于它或其父级的标记，以及明确添加到它或其父级的任何 额外关键字。

注册标记

为你的测试套件注册标记很简单

# content of pytest.toml
[pytest]
markers = ["webtest: mark a test as a webtest.", "slow: mark test as slow."]

可以通过在每一行定义一个标记来注册多个自定义标记，如上例所示。

你可以询问测试套件中存在哪些标记 - 该列表包括我们刚才定义的 webtest 和 slow 标记

$ pytest --markers
@pytest.mark.webtest: mark a test as a webtest.

@pytest.mark.slow: mark test as slow.

@pytest.mark.filterwarnings(warning): add a warning filter to the given test. see https://pytest.cn/en/stable/how-to/capture-warnings.html#pytest-mark-filterwarnings

@pytest.mark.skip(reason=None): skip the given test function with an optional reason. Example: skip(reason="no way of currently testing this") skips the test.

@pytest.mark.skipif(condition, ..., *, reason=...): skip the given test function if any of the conditions evaluate to True. Example: skipif(sys.platform == 'win32') skips the test if we are on the win32 platform. See https://pytest.cn/en/stable/reference/reference.html#pytest-mark-skipif

@pytest.mark.xfail(condition, ..., *, reason=..., run=True, raises=None, strict=strict_xfail): mark the test function as an expected failure if any of the conditions evaluate to True. Optionally specify a reason for better reporting and run=False if you don't even want to execute the test function. If only specific exception(s) are expected, you can list them in raises, and if the test fails in other ways, it will be reported as a true failure. See https://pytest.cn/en/stable/reference/reference.html#pytest-mark-xfail

@pytest.mark.parametrize(argnames, argvalues): call a test function multiple times passing in different arguments in turn. argvalues generally needs to be a list of values if argnames specifies only one name or a list of tuples of values if argnames specifies multiple names. Example: @parametrize('arg1', [1,2]) would lead to two calls of the decorated test function, one with arg1=1 and another with arg1=2.see https://pytest.cn/en/stable/how-to/parametrize.html for more info and examples.

@pytest.mark.usefixtures(fixturename1, fixturename2, ...): mark tests as needing all of the specified fixtures. see https://pytest.cn/en/stable/explanation/fixtures.html#usefixtures

@pytest.mark.tryfirst: mark a hook implementation function such that the plugin machinery will try to call it first/as early as possible. DEPRECATED, use @pytest.hookimpl(tryfirst=True) instead.

@pytest.mark.trylast: mark a hook implementation function such that the plugin machinery will try to call it last/as late as possible. DEPRECATED, use @pytest.hookimpl(trylast=True) instead.

有关如何从插件添加和使用标记的示例，请参阅 控制测试运行的自定义标记和命令行选项。

注意

建议显式注册标记，以便

• 在你的测试套件中有一个地方定义你的标记

• 通过 pytest --markers 查询现有标记会得到很好的输出

• 如果你使用 strict_markers 配置选项，函数标记中的拼写错误将被视为错误。

标记整个类或模块

你可以在类上使用 pytest.mark 装饰器，将标记应用于其所有测试方法

# content of test_mark_classlevel.py
import pytest


@pytest.mark.webtest
class TestClass:
    def test_startup(self):
        pass

    def test_startup_and_more(self):
        pass

这等同于直接将装饰器应用于这两个测试函数。

要在模块级别应用标记，请使用 pytestmark 全局变量

import pytest
pytestmark = pytest.mark.webtest

或者多个标记

pytestmark = [pytest.mark.webtest, pytest.mark.slowtest]

由于历史原因，在引入类装饰器之前，可以在测试类上设置 pytestmark 属性，如下所示

import pytest


class TestClass:
    pytestmark = pytest.mark.webtest

在使用 parametrize 时标记单个测试

使用 parametrize 时，应用标记会使其应用于每个单独的测试。但是，也可以将标记应用于单个测试实例

import pytest


@pytest.mark.foo
@pytest.mark.parametrize(
    ("n", "expected"), [(1, 2), pytest.param(1, 3, marks=pytest.mark.bar), (2, 3)]
)
def test_increment(n, expected):
    assert n + 1 == expected

在此示例中，标记 “foo” 将应用于三个测试中的每一个，而 “bar” 标记仅应用于第二个测试。Skip 和 xfail 标记也可以通过这种方式应用，请参阅 使用 parametrize 进行跳过/预期失败。

控制测试运行的自定义标记和命令行选项

插件可以提供自定义标记并据此实现特定的行为。这是一个独立的示例，它添加了一个命令行选项和一个参数化测试函数标记，以运行通过命名环境指定的测试

# content of conftest.py

import pytest


def pytest_addoption(parser):
    parser.addoption(
        "-E",
        action="store",
        metavar="NAME",
        help="only run tests matching the environment NAME.",
    )


def pytest_configure(config):
    # register an additional marker
    config.addinivalue_line(
        "markers", "env(name): mark test to run only on named environment"
    )


def pytest_runtest_setup(item):
    envnames = [mark.args[0] for mark in item.iter_markers(name="env")]
    if envnames:
        if item.config.getoption("-E") not in envnames:
            pytest.skip(f"test requires env in {envnames!r}")

使用此本地插件的测试文件

# content of test_someenv.py

import pytest


@pytest.mark.env("stage1")
def test_basic_db_operation():
    pass

以及指定与测试所需环境不同的环境的示例调用

$ pytest -E stage2
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-9.x.y, pluggy-1.x.y
rootdir: /home/sweet/project
collected 1 item

test_someenv.py s                                                    [100%]

============================ 1 skipped in 0.12s ============================

这里是一个指定了所需环境的示例

$ pytest -E stage1
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-9.x.y, pluggy-1.x.y
rootdir: /home/sweet/project
collected 1 item

test_someenv.py .                                                    [100%]

============================ 1 passed in 0.12s =============================

--markers 选项总是会为你提供可用标记的列表

$ pytest --markers
@pytest.mark.env(name): mark test to run only on named environment

@pytest.mark.filterwarnings(warning): add a warning filter to the given test. see https://pytest.cn/en/stable/how-to/capture-warnings.html#pytest-mark-filterwarnings

@pytest.mark.skip(reason=None): skip the given test function with an optional reason. Example: skip(reason="no way of currently testing this") skips the test.

@pytest.mark.skipif(condition, ..., *, reason=...): skip the given test function if any of the conditions evaluate to True. Example: skipif(sys.platform == 'win32') skips the test if we are on the win32 platform. See https://pytest.cn/en/stable/reference/reference.html#pytest-mark-skipif

@pytest.mark.xfail(condition, ..., *, reason=..., run=True, raises=None, strict=strict_xfail): mark the test function as an expected failure if any of the conditions evaluate to True. Optionally specify a reason for better reporting and run=False if you don't even want to execute the test function. If only specific exception(s) are expected, you can list them in raises, and if the test fails in other ways, it will be reported as a true failure. See https://pytest.cn/en/stable/reference/reference.html#pytest-mark-xfail

@pytest.mark.parametrize(argnames, argvalues): call a test function multiple times passing in different arguments in turn. argvalues generally needs to be a list of values if argnames specifies only one name or a list of tuples of values if argnames specifies multiple names. Example: @parametrize('arg1', [1,2]) would lead to two calls of the decorated test function, one with arg1=1 and another with arg1=2.see https://pytest.cn/en/stable/how-to/parametrize.html for more info and examples.

@pytest.mark.usefixtures(fixturename1, fixturename2, ...): mark tests as needing all of the specified fixtures. see https://pytest.cn/en/stable/explanation/fixtures.html#usefixtures

@pytest.mark.tryfirst: mark a hook implementation function such that the plugin machinery will try to call it first/as early as possible. DEPRECATED, use @pytest.hookimpl(tryfirst=True) instead.

@pytest.mark.trylast: mark a hook implementation function such that the plugin machinery will try to call it last/as late as possible. DEPRECATED, use @pytest.hookimpl(trylast=True) instead.

将可调用对象传递给自定义标记

下面是将在接下来的示例中使用的配置文件

# content of conftest.py
import sys


def pytest_runtest_setup(item):
    for marker in item.iter_markers(name="my_marker"):
        print(marker)
        sys.stdout.flush()

自定义标记可以设置其参数，即 args 和 kwargs 属性，既可以通过将其作为可调用对象调用，也可以使用 pytest.mark.MARKER_NAME.with_args 来定义。大多数情况下，这两种方法效果相同。

然而，如果作为唯一的单个位置参数有一个可调用对象且没有关键字参数，使用 pytest.mark.MARKER_NAME(c) 不会将 c 作为位置参数传递，而是用自定义标记装饰 c（参见 MarkDecorator）。幸运的是，pytest.mark.MARKER_NAME.with_args 可以解决这个问题

# content of test_custom_marker.py
import pytest


def hello_world(*args, **kwargs):
    return "Hello World"


@pytest.mark.my_marker.with_args(hello_world)
def test_with_args():
    pass

输出如下

$ pytest -q -s
Mark(name='my_marker', args=(<function hello_world at 0xdeadbeef0001>,), kwargs={})
.
1 passed in 0.12s

我们可以看到，自定义标记的参数集扩展了函数 hello_world。这就是将自定义标记创建为可调用对象（在后台调用 __call__）与使用 with_args 之间的关键区别。

读取从多个地方设置的标记

如果你在测试套件中大量使用标记，可能会遇到多次将同一标记应用于测试函数的情况。你可以从插件代码中读取所有这些设置。示例

# content of test_mark_three_times.py
import pytest

pytestmark = pytest.mark.glob("module", x=1)


@pytest.mark.glob("class", x=2)
class TestClass:
    @pytest.mark.glob("function", x=3)
    def test_something(self):
        pass

在这里，我们将标记 “glob” 三次应用于同一个测试函数。我们可以从 conftest 文件中像这样读取它

# content of conftest.py
import sys


def pytest_runtest_setup(item):
    for mark in item.iter_markers(name="glob"):
        print(f"glob args={mark.args} kwargs={mark.kwargs}")
        sys.stdout.flush()

让我们在不捕获输出的情况下运行它，看看会得到什么

$ pytest -q -s
glob args=('function',) kwargs={'x': 3}
glob args=('class',) kwargs={'x': 2}
glob args=('module',) kwargs={'x': 1}
.
1 passed in 0.12s

使用 pytest 标记平台特定的测试

假设你有一个为特定平台标记测试的测试套件，即 pytest.mark.darwin, pytest.mark.win32 等，并且你还有在所有平台上运行且没有特定标记的测试。如果你现在想要一种方法来仅运行特定平台的测试，可以使用以下插件

# content of conftest.py
#
import sys

import pytest

ALL = set("darwin linux win32".split())


def pytest_runtest_setup(item):
    supported_platforms = ALL.intersection(mark.name for mark in item.iter_markers())
    plat = sys.platform
    if supported_platforms and plat not in supported_platforms:
        pytest.skip(f"cannot run on platform {plat}")

那么如果为不同的平台指定了测试，它们将被跳过。让我们做一个小测试文件来看看它是怎样的

# content of test_plat.py

import pytest


@pytest.mark.darwin
def test_if_apple_is_evil():
    pass


@pytest.mark.linux
def test_if_linux_works():
    pass


@pytest.mark.win32
def test_if_win32_crashes():
    pass


def test_runs_everywhere():
    pass

然后你会看到两个测试被跳过，两个测试按预期执行

$ pytest -rs # this option reports skip reasons
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-9.x.y, pluggy-1.x.y
rootdir: /home/sweet/project
collected 4 items

test_plat.py s.s.                                                    [100%]

========================= short test summary info ==========================
SKIPPED [2] conftest.py:13: cannot run on platform linux
======================= 2 passed, 2 skipped in 0.12s =======================

注意，如果你通过标记命令行选项指定了平台，如下所示

$ pytest -m linux
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-9.x.y, pluggy-1.x.y
rootdir: /home/sweet/project
collected 4 items / 3 deselected / 1 selected

test_plat.py .                                                       [100%]

===================== 1 passed, 3 deselected in 0.12s ======================

那么未标记的测试将不会运行。因此，这是一种将运行限制在特定测试上的方法。

基于测试名称自动添加标记

如果你有一个测试套件，其中的测试函数名称指示了某种类型的测试，你可以实现一个钩子来自动定义标记，以便你可以使用 -m 选项。让我们看看这个测试模块

# content of test_module.py


def test_interface_simple():
    assert 0


def test_interface_complex():
    assert 0


def test_event_simple():
    assert 0


def test_something_else():
    assert 0

我们想要动态定义两个标记，并可以在 conftest.py 插件中完成

# content of conftest.py

import pytest


def pytest_collection_modifyitems(items):
    for item in items:
        if "interface" in item.nodeid:
            item.add_marker(pytest.mark.interface)
        elif "event" in item.nodeid:
            item.add_marker(pytest.mark.event)

我们现在可以使用 -m 选项 来选择一组

$ pytest -m interface --tb=short
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-9.x.y, pluggy-1.x.y
rootdir: /home/sweet/project
collected 4 items / 2 deselected / 2 selected

test_module.py FF                                                    [100%]

================================= FAILURES =================================
__________________________ test_interface_simple ___________________________
test_module.py:4: in test_interface_simple
    assert 0
E   assert 0
__________________________ test_interface_complex __________________________
test_module.py:8: in test_interface_complex
    assert 0
E   assert 0
========================= short test summary info ==========================
FAILED test_module.py::test_interface_simple - assert 0
FAILED test_module.py::test_interface_complex - assert 0
===================== 2 failed, 2 deselected in 0.12s ======================

或者选择 “event” 和 “interface” 两种测试

$ pytest -m "interface or event" --tb=short
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-9.x.y, pluggy-1.x.y
rootdir: /home/sweet/project
collected 4 items / 1 deselected / 3 selected

test_module.py FFF                                                   [100%]

================================= FAILURES =================================
__________________________ test_interface_simple ___________________________
test_module.py:4: in test_interface_simple
    assert 0
E   assert 0
__________________________ test_interface_complex __________________________
test_module.py:8: in test_interface_complex
    assert 0
E   assert 0
____________________________ test_event_simple _____________________________
test_module.py:12: in test_event_simple
    assert 0
E   assert 0
========================= short test summary info ==========================
FAILED test_module.py::test_interface_simple - assert 0
FAILED test_module.py::test_interface_complex - assert 0
FAILED test_module.py::test_event_simple - assert 0
===================== 3 failed, 1 deselected in 0.12s ======================