Quality

Testing workflow

Use fake_initialization_context, fake_execution_context, TestClock, ProgressCollector, and InMemoryArtifactStore for deterministic tests.

import pytest
from vega_capability_sdk import CapabilityExecutionRequest
from vega_capability_sdk.testing import ProviderContractHarness

@pytest.mark.asyncio
async def test_provider_contract() -> None:
    provider = EchoProvider()
    request = CapabilityExecutionRequest(
        execution_id="e-1", capability_id="example.echo",
        provider_id="example.echo.provider", action_id="example.echo.run",
        action_version="1.0.0", inputs={"text": "hello"},
    )
    report = await ProviderContractHarness().run(provider, request)
    report.raise_for_failures()

Validation and static analysis

Run package validation before tests:

vega-capability validate . --json
pytest
ruff check src tests
mypy src

The validator checks package structure, manifest, schemas, paths, resources, execution declarations, contracts, and SDK compatibility. It does not import or start the Provider, validate live hardware, or install dependencies.

Debugging and observability

Use context-bound structured logging. Include non-secret identifiers and backend status; never include raw secret values. Health checks should be side-effect free. Test cancellation before I/O and during long-running work. Test recovery probes for each declared recovery behavior.

Reliability and performance

Measure latency, memory, vendor timeouts, cancellation response, and artifact size for your actual workload. Test resource contention and repeated startup/shutdown. The conformance harness is a protocol baseline, not proof of physical safety, vendor reliability, or Runtime integration correctness.