Pārlūkot izejas kodu

#22486: Pre-release QA (#22862)

Normalize RQ timeout values before validating global and per-webhook
timeouts, including duration strings and RQ's default and unlimited values.

Improve timeout logging and visibility in the UI and documentation, raise
the default webhook timeout to 60 seconds, and add coverage for the new
validation and filtering behavior.
Jeremy Stretch 2 nedēļas atpakaļ
vecāks
revīzija
d642a43121

+ 14 - 3
docs/configuration/miscellaneous.md

@@ -277,7 +277,10 @@ This is a wrapper for passing global configuration parameters to [Django RQ](htt
 
 Default: `300`
 
-The maximum execution time of a background task (such as running a custom script), in seconds.
+The maximum execution time of a background task (such as running a custom script), in seconds. This may also be expressed as a duration string such as `1h` or `30m`, which NetBox normalizes to seconds when comparing it against webhook timeouts. Set this to `-1` to disable the job timeout entirely.
+
+!!! note
+    A value of zero (or `None`) does not disable the timeout: RQ falls back to its own default of 180 seconds, and NetBox validates webhook timeouts against that value accordingly.
 
 ---
 
@@ -311,6 +314,14 @@ The base unit for RAM sizes. Set this to `1024` to use binary prefixes (MiB, GiB
 
 ## WEBHOOK_DEFAULT_TIMEOUT
 
-Default: `30`
+Default: `60`
+
+The default maximum time (in seconds) to wait for a response when sending a webhook. This value is used for any webhook which does not define its own timeout. Keeping this below [`RQ_DEFAULT_TIMEOUT`](#rq_default_timeout) gives an unresponsive receiver a chance to be cut off by the request timeout rather than by termination of the background job.
 
-The default maximum time (in seconds) to wait for a response when sending a webhook. This value is used for any webhook which does not define its own timeout. Setting a value lower than [`RQ_DEFAULT_TIMEOUT`](#rq_default_timeout) prevents an unresponsive receiver from holding a background worker open for the full duration of the job.
+This value must be an integer between 1 and 3600, and must be less than `RQ_DEFAULT_TIMEOUT`; NetBox will refuse to start otherwise. The same upper bound is enforced on the per-webhook [timeout](../models/extras/webhook.md#timeout) field.
+
+!!! warning "Upgrading"
+    If you have lowered `RQ_DEFAULT_TIMEOUT` to 60 seconds or less and have not set `WEBHOOK_DEFAULT_TIMEOUT`, NetBox will not start until you set `WEBHOOK_DEFAULT_TIMEOUT` to a value below your job timeout.
+
+!!! note
+    The timeout is applied separately to establishing the connection and to waiting for data, rather than to the request as a whole. A receiver which responds slowly but continuously can therefore keep a request open for longer than the configured value. `RQ_DEFAULT_TIMEOUT` remains the ultimate upper bound on how long a webhook job can occupy a worker.

+ 7 - 0
docs/models/extras/webhook.md

@@ -86,6 +86,13 @@ The file path to a particular certificate authority (CA) file to use when valida
 
 The maximum time (in seconds) to wait for a response from the receiver before the request is considered failed. If left blank, the global [`WEBHOOK_DEFAULT_TIMEOUT`](../../configuration/miscellaneous.md#webhook_default_timeout) configuration value is used.
 
+The timeout must be less than [`RQ_DEFAULT_TIMEOUT`](../../configuration/miscellaneous.md#rq_default_timeout) (300 seconds by default), and NetBox will refuse to save a webhook which violates this. The background job timeout is a hard ceiling on how long a webhook request can run, so a value at or above it leaves no room for the request's own timeout to apply.
+
+!!! note
+    Staying below the job timeout makes it *likely*, but does not guarantee, that the request times out on its own. The timeout is applied separately to establishing the connection and to waiting for data, rather than to the request as a whole, so a receiver which stalls at both stages — or which responds slowly but continuously — can still outlast the job timeout and be terminated by the worker instead.
+
+When a request does time out, the failure is recorded by the `netbox.webhooks` logger and the background job is marked as failed.
+
 ## Context Data
 
 The following context variables are available to the text and link templates.

+ 13 - 1
netbox/extras/forms/filtersets.py

@@ -316,7 +316,9 @@ class WebhookFilterForm(OwnerFilterMixin, NetBoxModelFilterSetForm):
     model = Webhook
     fieldsets = (
         FieldSet('q', 'filter_id', 'tag'),
-        FieldSet('payload_url', 'http_method', 'http_content_type', name=_('Attributes')),
+        FieldSet(
+            'payload_url', 'http_method', 'http_content_type', 'timeout__gte', 'timeout__lte', name=_('Attributes')
+        ),
         FieldSet('owner_group_id', 'owner_id', name=_('Ownership')),
     )
     http_content_type = forms.CharField(
@@ -332,6 +334,16 @@ class WebhookFilterForm(OwnerFilterMixin, NetBoxModelFilterSetForm):
         required=False,
         label=_('HTTP method')
     )
+    timeout__gte = forms.IntegerField(
+        required=False,
+        min_value=1,
+        label=_('Minimum timeout (seconds)')
+    )
+    timeout__lte = forms.IntegerField(
+        required=False,
+        min_value=1,
+        label=_('Maximum timeout (seconds)')
+    )
     tag = TagFilterField(model)
 
 

+ 21 - 4
netbox/extras/models/models.py

@@ -5,12 +5,14 @@ from pathlib import Path
 from django.conf import settings
 from django.contrib.contenttypes.fields import GenericForeignKey, GenericRelation
 from django.contrib.postgres.fields import ArrayField
-from django.core.validators import MaxValueValidator, MinValueValidator, ValidationError
+from django.core.exceptions import ValidationError
+from django.core.validators import MaxValueValidator, MinValueValidator
 from django.db import models
 from django.urls import reverse
 from django.utils import timezone
 from django.utils.html import escape
 from django.utils.safestring import mark_safe
+from django.utils.text import format_lazy
 from django.utils.translation import gettext_lazy as _
 from rest_framework.utils.encoders import JSONEncoder
 
@@ -34,6 +36,7 @@ from netbox.models.features import (
     has_feature,
 )
 from netbox.models.mixins import OwnerMixin
+from netbox.settings_utils import parse_job_timeout
 from utilities.html import clean_html
 from utilities.jinja2 import render_jinja2, sanitize_http_header
 from utilities.querydict import dict_to_querydict
@@ -287,9 +290,12 @@ class Webhook(CustomFieldsMixin, ExportTemplatesMixin, TagsMixin, OwnerMixin, Ch
             MinValueValidator(1),
             MaxValueValidator(3600),
         ),
-        help_text=_(
-            "The maximum time (in seconds) to wait for a response before failing the request. Leave blank to use "
-            "the system default (WEBHOOK_DEFAULT_TIMEOUT)."
+        help_text=format_lazy(
+            _(
+                "The maximum time (in seconds) to wait for a response before failing the request. Leave blank to use "
+                "the system default ({default_timeout} seconds)."
+            ),
+            default_timeout=settings.WEBHOOK_DEFAULT_TIMEOUT
         )
     )
     events = GenericRelation(
@@ -322,6 +328,17 @@ class Webhook(CustomFieldsMixin, ExportTemplatesMixin, TagsMixin, OwnerMixin, Ch
                 'ca_file_path': _('Do not specify a CA certificate file if SSL verification is disabled.')
             })
 
+        # A timeout which meets or exceeds the background job timeout leaves no room for the request's own timeout
+        # to apply: the worker will terminate the job first. (Staying below the job timeout does not guarantee that
+        # the request times out on its own, as the timeout applies separately to connecting and to reading data.)
+        job_timeout = parse_job_timeout(settings.RQ_DEFAULT_TIMEOUT)
+        if self.timeout is not None and job_timeout is not None and self.timeout >= job_timeout:
+            raise ValidationError({
+                'timeout': _(
+                    "Timeout must be less than the background job timeout ({timeout} seconds)."
+                ).format(timeout=job_timeout)
+            })
+
     def render_headers(self, context):
         """
         Render additional_headers and return a dict of Header: Value pairs.

+ 3 - 0
netbox/extras/tables/tables.py

@@ -491,6 +491,9 @@ class WebhookTable(NetBoxTable):
     ssl_verification = columns.BooleanColumn(
         verbose_name=_('SSL Verification'),
     )
+    timeout = tables.Column(
+        verbose_name=_('Timeout (sec)'),
+    )
     owner = tables.Column(
         linkify=True,
         verbose_name=_('Owner')

+ 67 - 1
netbox/extras/tests/test_event_rules.py

@@ -6,10 +6,11 @@ from unittest import skipIf
 from unittest.mock import Mock, patch
 
 import django_rq
+import requests
 from django.conf import settings
 from django.core.exceptions import ImproperlyConfigured, ValidationError
 from django.http import HttpResponse
-from django.test import RequestFactory, TestCase, tag
+from django.test import RequestFactory, TestCase, override_settings, tag
 from django.urls import reverse
 from PIL import Image
 from requests import Session
@@ -508,6 +509,71 @@ class EventRuleTestCase(RQQueueTestMixin, APITestCase):
         with patch.object(Session, 'send', dummy_send):
             send_webhook(**job.kwargs)
 
+    @override_settings(RQ_DEFAULT_TIMEOUT=10)
+    def test_send_webhook_timeout_exceeding_job_timeout_is_logged(self):
+        """
+        A timeout which meets or exceeds the background job timeout should be logged as a warning. This can
+        occur when RQ_DEFAULT_TIMEOUT has been lowered after the webhook was saved, which Webhook.clean()
+        cannot catch.
+        """
+        webhook = Webhook.objects.get(name='Webhook 1')
+        webhook.timeout = 30
+        webhook.save()
+
+        request = RequestFactory().get(reverse('dcim:site_add'))
+        request.id = uuid.uuid4()
+        request.user = self.user
+
+        webhooks_queue = {}
+        site = Site.objects.create(name='Site 1', slug='site-1')
+        enqueue_event(
+            webhooks_queue,
+            instance=site,
+            request=request,
+            event_type=OBJECT_CREATED,
+        )
+        flush_events(list(webhooks_queue.values()))
+
+        job = self.queue.jobs[0]
+        with patch.object(Session, 'send', lambda _, request, **kwargs: HttpResponse()):
+            with self.assertLogs('netbox.webhooks', level='WARNING') as cm:
+                send_webhook(**job.kwargs)
+
+        self.assertIn(
+            'Webhook timeout (30 seconds) is not less than the background job timeout (10 seconds)',
+            '\n'.join(cm.output)
+        )
+
+    def test_send_webhook_timeout_is_logged(self):
+        """
+        A request which times out should be logged as an error before the exception is re-raised, so that the
+        failure is discoverable without resorting to the RQ worker's traceback.
+        """
+        def timing_out_send(_, request, **kwargs):
+            raise requests.exceptions.ConnectTimeout('Connection timed out')
+
+        request = RequestFactory().get(reverse('dcim:site_add'))
+        request.id = uuid.uuid4()
+        request.user = self.user
+
+        webhooks_queue = {}
+        site = Site.objects.create(name='Site 1', slug='site-1')
+        enqueue_event(
+            webhooks_queue,
+            instance=site,
+            request=request,
+            event_type=OBJECT_CREATED,
+        )
+        flush_events(list(webhooks_queue.values()))
+
+        job = self.queue.jobs[0]
+        with patch.object(Session, 'send', timing_out_send):
+            with self.assertLogs('netbox.webhooks', level='ERROR') as cm:
+                with self.assertRaises(requests.exceptions.Timeout):
+                    send_webhook(**job.kwargs)
+
+        self.assertIn(f'timed out after {settings.WEBHOOK_DEFAULT_TIMEOUT} seconds', cm.output[0])
+
     def test_job_completed_webhook_without_request(self):
         """
         Ensure job_end event processing can enqueue a webhook even when the EventContext

+ 7 - 0
netbox/extras/tests/test_filtersets.py

@@ -285,6 +285,13 @@ class WebhookTestCase(TestCase, BaseFilterSetTestMixin):
         params = {'timeout': [10, 20]}
         self.assertEqual(self.filterset(params, self.queryset).qs.count(), 2)
 
+    def test_timeout_range(self):
+        # Backs the minimum/maximum timeout fields exposed by WebhookFilterForm
+        params = {'timeout__gte': [20]}
+        self.assertEqual(self.filterset(params, self.queryset).qs.count(), 2)
+        params = {'timeout__lte': [20]}
+        self.assertEqual(self.filterset(params, self.queryset).qs.count(), 2)
+
 
 class EventRuleTestCase(TestCase, BaseFilterSetTestMixin):
     queryset = EventRule.objects.all()

+ 69 - 1
netbox/extras/tests/test_models.py

@@ -13,10 +13,11 @@ from django.core.files.storage import Storage
 from django.core.files.uploadedfile import SimpleUploadedFile
 from django.db import connection
 from django.forms import ValidationError
-from django.test import TestCase, tag
+from django.test import TestCase, override_settings, tag
 from django.test.utils import CaptureQueriesContext
 from jinja2 import DebugUndefined, StrictUndefined, TemplateError, TemplateSyntaxError, UndefinedError
 from PIL import Image
+from rq.queue import Queue
 
 from core.events import OBJECT_CREATED
 from core.models import AutoSyncRecord, DataSource, ObjectType
@@ -1806,3 +1807,70 @@ class JinjaEnvironmentParamsIntegrationTestCase(TestCase):
         # ConfigTemplate always forces autoescape off (#22652).
         template = self._make_template({})
         self.assertEqual(template.get_environment_params(), {'autoescape': False})
+
+
+@override_settings(RQ_DEFAULT_TIMEOUT=300)
+class WebhookTestCase(TestCase):
+
+    def test_timeout_must_be_less_than_job_timeout(self):
+        """
+        A timeout at or above RQ_DEFAULT_TIMEOUT leaves no room for the request's own timeout to apply, and
+        is rejected.
+        """
+        webhook = Webhook(name='Webhook 1', payload_url='http://localhost:9000/')
+
+        for timeout in (300, 301):
+            webhook.timeout = timeout
+            with self.assertRaises(ValidationError):
+                webhook.full_clean()
+
+    def test_timeout_below_job_timeout_is_valid(self):
+        webhook = Webhook(name='Webhook 1', payload_url='http://localhost:9000/', timeout=299)
+        webhook.full_clean()
+
+    def test_null_timeout_is_valid(self):
+        webhook = Webhook(name='Webhook 1', payload_url='http://localhost:9000/')
+        webhook.full_clean()
+
+    @override_settings(RQ_DEFAULT_TIMEOUT='1h')
+    def test_job_timeout_duration_string_is_validated(self):
+        """
+        RQ also accepts a string timeout such as "1h", which must be normalized before comparison rather
+        than bypassing the check.
+        """
+        webhook = Webhook(name='Webhook 1', payload_url='http://localhost:9000/', timeout=3600)
+        with self.assertRaises(ValidationError):
+            webhook.full_clean()
+
+        webhook.timeout = 3599
+        webhook.full_clean()
+
+    @override_settings(RQ_DEFAULT_TIMEOUT='60')
+    def test_job_timeout_numeric_string_is_validated(self):
+        webhook = Webhook(name='Webhook 1', payload_url='http://localhost:9000/', timeout=60)
+        with self.assertRaises(ValidationError):
+            webhook.full_clean()
+
+        webhook.timeout = 59
+        webhook.full_clean()
+
+    @override_settings(RQ_DEFAULT_TIMEOUT=-1)
+    def test_unbounded_job_timeout_skips_validation(self):
+        """
+        A negative RQ timeout (-1) disables RQ's death penalty, so there is no job timeout to validate against.
+        """
+        webhook = Webhook(name='Webhook 1', payload_url='http://localhost:9000/', timeout=3600)
+        webhook.full_clean()
+
+    @override_settings(RQ_DEFAULT_TIMEOUT=0)
+    def test_zero_job_timeout_is_validated_against_queue_default(self):
+        """
+        A zero (or absent) RQ timeout is not unbounded: RQ falls back to the queue's own default, which the
+        webhook timeout must still stay below.
+        """
+        webhook = Webhook(name='Webhook 1', payload_url='http://localhost:9000/', timeout=Queue.DEFAULT_TIMEOUT)
+        with self.assertRaises(ValidationError):
+            webhook.full_clean()
+
+        webhook.timeout = Queue.DEFAULT_TIMEOUT - 1
+        webhook.full_clean()

+ 13 - 1
netbox/extras/ui/panels.py

@@ -1,3 +1,4 @@
+from django.conf import settings
 from django.contrib.contenttypes.models import ContentType
 from django.utils.translation import gettext_lazy as _
 
@@ -333,6 +334,17 @@ class WebhookPanel(panels.ObjectAttributesPanel):
     description = attrs.TextAttr('description')
 
 
+class WebhookTimeoutAttr(attrs.TextAttr):
+    """
+    Render a webhook's timeout. Webhooks which do not define their own timeout fall back to the globally
+    configured default, which is displayed (and annotated as such) in place of an empty value.
+    """
+    def get_value(self, obj):
+        if obj.timeout is not None:
+            return _('{timeout} seconds').format(timeout=obj.timeout)
+        return _('{timeout} seconds (default)').format(timeout=settings.WEBHOOK_DEFAULT_TIMEOUT)
+
+
 class WebhookHTTPPanel(panels.ObjectAttributesPanel):
     title = _('HTTP Request')
 
@@ -340,7 +352,7 @@ class WebhookHTTPPanel(panels.ObjectAttributesPanel):
     payload_url = attrs.TextAttr('payload_url', label=_('Payload URL'), style='font-monospace')
     http_content_type = attrs.TextAttr('http_content_type', label=_('HTTP content type'))
     secret = attrs.TextAttr('secret')
-    timeout = attrs.TextAttr('timeout')
+    timeout = WebhookTimeoutAttr('timeout')
 
 
 class WebhookSSLPanel(panels.ObjectAttributesPanel):

+ 14 - 1
netbox/extras/webhooks.py

@@ -8,6 +8,7 @@ from django_rq import job
 from jinja2.exceptions import TemplateError
 
 from netbox.registry import registry
+from netbox.settings_utils import parse_job_timeout
 from utilities.proxy import resolve_proxies
 from utilities.request import get_safe_request_context
 
@@ -116,13 +117,25 @@ def send_webhook(event_rule, object_type, event_type, data, timestamp, request=N
     # Determine the request timeout, preferring the webhook-specific value over the global default
     timeout = webhook.timeout if webhook.timeout is not None else settings.WEBHOOK_DEFAULT_TIMEOUT
 
+    # Webhook.clean() enforces this when the webhook is saved, but RQ_DEFAULT_TIMEOUT may have been lowered since.
+    job_timeout = parse_job_timeout(settings.RQ_DEFAULT_TIMEOUT)
+    if job_timeout is not None and timeout >= job_timeout:
+        logger.warning(
+            f"Webhook timeout ({timeout} seconds) is not less than the background job timeout ({job_timeout} "
+            f"seconds); the job may be terminated before the request can time out."
+        )
+
     # Send the request
     with requests.Session() as session:
         session.verify = webhook.ssl_verification
         if webhook.ca_file_path:
             session.verify = webhook.ca_file_path
         proxies = resolve_proxies(url=url, context={'client': webhook})
-        response = session.send(prepared_request, proxies=proxies, timeout=timeout)
+        try:
+            response = session.send(prepared_request, proxies=proxies, timeout=timeout)
+        except requests.exceptions.Timeout:
+            logger.error(f"Request to {url} timed out after {timeout} seconds")
+            raise
 
     if 200 <= response.status_code <= 299:
         logger.info(f"Request succeeded; response status {response.status_code}")

+ 4 - 3
netbox/netbox/configuration_example.py

@@ -224,9 +224,10 @@ RQ_DEFAULT_TIMEOUT = 300
 # SCRIPTS_ROOT = '/path/to/netbox/scripts'
 
 # The default maximum time (in seconds) to wait for a response when sending a webhook, unless overridden on the
-# individual webhook. This prevents an unresponsive receiver from holding a background worker open for the full
-# RQ_DEFAULT_TIMEOUT duration.
-WEBHOOK_DEFAULT_TIMEOUT = 30
+# individual webhook. Keeping this below RQ_DEFAULT_TIMEOUT gives an unresponsive receiver a chance to be cut off
+# by the request timeout rather than by termination of the background job. This must be less than
+# RQ_DEFAULT_TIMEOUT.
+WEBHOOK_DEFAULT_TIMEOUT = 60
 
 # The name to use for the session cookie.
 SESSION_COOKIE_NAME = 'sessionid'

+ 10 - 6
netbox/netbox/settings.py

@@ -18,7 +18,14 @@ from netbox.config import PARAMS as CONFIG_PARAMS
 from netbox.constants import RQ_QUEUE_DEFAULT, RQ_QUEUE_HIGH, RQ_QUEUE_LOW
 from netbox.plugins import PluginConfig
 from netbox.registry import registry
-from netbox.settings_utils import get_configuration_dir, load_configuration, resolve_install_paths, secret_key_hint
+from netbox.settings_utils import (
+    get_configuration_dir,
+    load_configuration,
+    parse_job_timeout,
+    resolve_install_paths,
+    secret_key_hint,
+    validate_webhook_default_timeout,
+)
 from utilities.release import load_release_data
 from utilities.security import validate_peppers
 from utilities.string import trailing_slash
@@ -220,11 +227,8 @@ STORAGE_CONFIG = getattr(configuration, 'STORAGE_CONFIG', None)
 STORAGES = getattr(configuration, 'STORAGES', {})
 TIME_ZONE = getattr(configuration, 'TIME_ZONE', 'UTC')
 TRANSLATION_ENABLED = getattr(configuration, 'TRANSLATION_ENABLED', True)
-WEBHOOK_DEFAULT_TIMEOUT = getattr(configuration, 'WEBHOOK_DEFAULT_TIMEOUT', 30)
-if not isinstance(WEBHOOK_DEFAULT_TIMEOUT, int) or not 1 <= WEBHOOK_DEFAULT_TIMEOUT <= 3600:
-    raise ImproperlyConfigured(
-        f"WEBHOOK_DEFAULT_TIMEOUT must be an integer between 1 and 3600 (found {WEBHOOK_DEFAULT_TIMEOUT!r})"
-    )
+WEBHOOK_DEFAULT_TIMEOUT = getattr(configuration, 'WEBHOOK_DEFAULT_TIMEOUT', 60)
+validate_webhook_default_timeout(WEBHOOK_DEFAULT_TIMEOUT, parse_job_timeout(RQ_DEFAULT_TIMEOUT))
 DISK_BASE_UNIT = getattr(configuration, 'DISK_BASE_UNIT', 1000)
 if DISK_BASE_UNIT not in [1000, 1024]:
     raise ImproperlyConfigured(f"DISK_BASE_UNIT must be 1000 or 1024 (found {DISK_BASE_UNIT})")

+ 47 - 0
netbox/netbox/settings_utils.py

@@ -8,14 +8,19 @@ import warnings
 from typing import NamedTuple
 
 from django.core.exceptions import ImproperlyConfigured
+from rq.exceptions import TimeoutFormatError
+from rq.queue import Queue
+from rq.utils import parse_timeout
 
 __all__ = (
     'InstallPaths',
     'get_configuration_dir',
     'load_configuration',
     'load_ldap_config',
+    'parse_job_timeout',
     'resolve_install_paths',
     'secret_key_hint',
+    'validate_webhook_default_timeout',
 )
 
 
@@ -72,6 +77,48 @@ def secret_key_hint(install_mode, base_dir):
     return f'python {base_dir}/generate_secret_key.py'
 
 
+def parse_job_timeout(value):
+    """Normalize an RQ job timeout (i.e. RQ_DEFAULT_TIMEOUT) to a number of seconds.
+
+    RQ accepts a timeout as an integer, as a numeric string, or as a duration string such as
+    "1h", so its own parser is used to arrive at a value which can be compared against webhook
+    timeouts. A negative timeout (-1 by convention) disables RQ's death penalty; that is reported
+    as None, meaning that job execution is unbounded. An absent or zero timeout is *not* unbounded:
+    RQ falls back to the queue's own default, which is reported in its place.
+    """
+    try:
+        timeout = parse_timeout(value)
+    except (TimeoutFormatError, TypeError):
+        raise ImproperlyConfigured(
+            f"RQ_DEFAULT_TIMEOUT must be a number of seconds or a duration string such as '1h' "
+            f"(found {value!r})"
+        )
+    if timeout is None or timeout == 0:
+        # Queue treats a null or zero default timeout as unset and substitutes its class default.
+        return Queue.DEFAULT_TIMEOUT
+    if timeout < 0:
+        return None
+    return timeout
+
+
+def validate_webhook_default_timeout(timeout, job_timeout):
+    """Validate WEBHOOK_DEFAULT_TIMEOUT, including against the background job timeout.
+
+    job_timeout is the normalized RQ_DEFAULT_TIMEOUT (see parse_job_timeout()), or None if job
+    execution is unbounded. A webhook timeout which meets or exceeds the job timeout leaves no
+    room for the request's own timeout to apply, as the worker will terminate the job first.
+    """
+    if not isinstance(timeout, int) or not 1 <= timeout <= 3600:
+        raise ImproperlyConfigured(
+            f"WEBHOOK_DEFAULT_TIMEOUT must be an integer between 1 and 3600 (found {timeout!r})"
+        )
+    if job_timeout is not None and timeout >= job_timeout:
+        raise ImproperlyConfigured(
+            f"WEBHOOK_DEFAULT_TIMEOUT ({timeout}) must be less than RQ_DEFAULT_TIMEOUT ({job_timeout} seconds), "
+            f"which caps the total runtime of the background job."
+        )
+
+
 def _import_module(name):
     """Import a configuration module by dotted path.
 

+ 70 - 0
netbox/netbox/tests/test_settings_utils.py

@@ -7,6 +7,7 @@ from unittest.mock import patch
 from django.conf import settings as django_settings
 from django.core.exceptions import ImproperlyConfigured
 from django.test import SimpleTestCase
+from rq.queue import Queue
 
 from netbox import settings_utils
 
@@ -211,6 +212,75 @@ class SecretKeyHintTest(SimpleTestCase):
         )
 
 
+class ParseJobTimeoutTest(SimpleTestCase):
+    """parse_job_timeout() normalizes RQ_DEFAULT_TIMEOUT to a comparable number of seconds."""
+
+    def test_integer_is_returned_unchanged(self):
+        self.assertEqual(settings_utils.parse_job_timeout(300), 300)
+
+    def test_numeric_string_is_coerced(self):
+        self.assertEqual(settings_utils.parse_job_timeout('300'), 300)
+
+    def test_duration_string_is_normalized(self):
+        self.assertEqual(settings_utils.parse_job_timeout('1h'), 3600)
+        self.assertEqual(settings_utils.parse_job_timeout('30m'), 1800)
+        self.assertEqual(settings_utils.parse_job_timeout('45s'), 45)
+
+    def test_absent_or_zero_timeout_falls_back_to_queue_default(self):
+        # RQ does not treat a null or zero default timeout as unlimited: Queue substitutes its own
+        # default, which remains a real ceiling on job execution.
+        for value in (None, 0, '0'):
+            with self.subTest(value=value):
+                self.assertEqual(settings_utils.parse_job_timeout(value), Queue.DEFAULT_TIMEOUT)
+
+    def test_negative_timeout_is_unbounded(self):
+        # -1 is RQ's documented infinite timeout; it disables the death penalty, so there is no
+        # ceiling to compare against.
+        self.assertIsNone(settings_utils.parse_job_timeout(-1))
+        self.assertIsNone(settings_utils.parse_job_timeout('-1'))
+
+    def test_invalid_value_raises(self):
+        for value in ('1x', 'abc', [300]):
+            with self.subTest(value=value):
+                with self.assertRaisesMessage(ImproperlyConfigured, 'RQ_DEFAULT_TIMEOUT'):
+                    settings_utils.parse_job_timeout(value)
+
+
+class ValidateWebhookDefaultTimeoutTest(SimpleTestCase):
+    """validate_webhook_default_timeout() is the startup check applied to WEBHOOK_DEFAULT_TIMEOUT."""
+
+    def test_valid_timeout_below_job_timeout(self):
+        settings_utils.validate_webhook_default_timeout(60, 300)
+
+    def test_timeout_at_or_above_job_timeout_raises(self):
+        for timeout in (300, 301):
+            with self.subTest(timeout=timeout):
+                with self.assertRaisesMessage(ImproperlyConfigured, 'must be less than RQ_DEFAULT_TIMEOUT'):
+                    settings_utils.validate_webhook_default_timeout(timeout, 300)
+
+    def test_normalized_job_timeout_is_enforced(self):
+        # A duration string such as "1h" must be normalized by the caller and enforced like any other value.
+        job_timeout = settings_utils.parse_job_timeout('1h')
+        with self.assertRaises(ImproperlyConfigured):
+            settings_utils.validate_webhook_default_timeout(3600, job_timeout)
+        settings_utils.validate_webhook_default_timeout(3599, job_timeout)
+
+    def test_unbounded_job_timeout_skips_comparison(self):
+        settings_utils.validate_webhook_default_timeout(3600, None)
+
+    def test_out_of_range_timeout_raises(self):
+        for timeout in (0, 3601):
+            with self.subTest(timeout=timeout):
+                with self.assertRaisesMessage(ImproperlyConfigured, 'between 1 and 3600'):
+                    settings_utils.validate_webhook_default_timeout(timeout, None)
+
+    def test_non_integer_timeout_raises(self):
+        for timeout in ('60', 60.5, None):
+            with self.subTest(timeout=timeout):
+                with self.assertRaisesMessage(ImproperlyConfigured, 'must be an integer'):
+                    settings_utils.validate_webhook_default_timeout(timeout, 300)
+
+
 class LoadLdapConfigTest(SimpleTestCase):
     def test_loads_sibling_ldap_config(self):
         with tempfile.TemporaryDirectory() as conf_dir: