[Bug]: Tests collection fails without internet connection #3400
Description
Steps to reproduce
- Disable internet connection
- Run
pytest srs/testsand get the error:
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='api.runpod.io', port=443): Max retries exceeded with url: /graphql (Caused by NameResolutionError("<urllib3.connection.HTTPSConnection object at 0x13908b680>: Failed to resolve 'api.runpod.io' ([Errno 8] nodename nor servname provided, or not known)"))
Actual behaviour
Although network connections are disallowed for tests (#2111), they do take place during test collection when importing modules. In this case, gpuhunt makes a call to runpod API on import, so tests fail if there are no internet connection (or runpod is down). The problem started to affect dstack after #3304.
Moreover, dstack server won't start if runpod is not available.
It found it during a short period when runpod api was not available – couldn't run tests.
Expected behaviour
No response
dstack version
master
Server logs
Additional information
src/tests/_internal/server/conftest.py:7: in <module>
from dstack._internal.server.main import app
src/dstack/_internal/server/main.py:1: in <module>
from dstack._internal.server.app import create_app, register_routes
src/dstack/_internal/server/app.py:24: in <module>
from dstack._internal.server.background import start_background_tasks
src/dstack/_internal/server/background/__init__.py:5: in <module>
from dstack._internal.server.background.tasks.process_compute_groups import process_compute_groups
src/dstack/_internal/server/background/tasks/process_compute_groups.py:18: in <module>
from dstack._internal.server.services import backends as backends_services
src/dstack/_internal/server/services/backends/__init__.py:16: in <module>
from dstack._internal.core.backends.configurators import (
src/dstack/_internal/core/backends/configurators.py:116: in <module>
from dstack._internal.core.backends.runpod.configurator import RunpodConfigurator
src/dstack/_internal/core/backends/runpod/configurator.py:8: in <module>
from dstack._internal.core.backends.runpod import api_client
src/dstack/_internal/core/backends/runpod/api_client.py:6: in <module>
from gpuhunt.providers.runpod import RunpodProvider
.venv/lib/python3.12/site-packages/gpuhunt/providers/runpod.py:286: in <module>
GPU_MAP = get_gpu_map()
.venv/lib/python3.12/site-packages/gpuhunt/providers/runpod.py:242: in get_gpu_map
response = make_request(payload_gpus)
.venv/lib/python3.12/site-packages/gpuhunt/providers/runpod.py:233: in make_request
resp = requests.post(API_URL, json=payload, timeout=10)
.venv/lib/python3.12/site-packages/requests/api.py:115: in post
return request("post", url, data=data, json=json, **kwargs)
.venv/lib/python3.12/site-packages/requests/api.py:59: in request
return session.request(method=method, url=url, **kwargs)
.venv/lib/python3.12/site-packages/requests/sessions.py:589: in request
resp = self.send(prep, **send_kwargs)
.venv/lib/python3.12/site-packages/requests/sessions.py:703: in send
r = adapter.send(request, **kwargs)
.venv/lib/python3.12/site-packages/requests/adapters.py:677: in send
raise ConnectionError(e, request=request)
E requests.exceptions.ConnectionError: HTTPSConnectionPool(host='api.runpod.io', port=443): Max retries exceeded with url: /graphql (Caused by NameResolutionError("<urllib3.connection.HTTPSConnection object at 0x13908b680>: Failed to resolve 'api.runpod.io' ([Errno 8] nodename nor servname provided, or not known)"))