Catch an AI key without credit or model access in the connection test (#563)

* Catch an AI key without credit or model access in the connection test

An OpenAI account with no credit answers every request with the same 429
as a rate limit, and a key that may not use a model with a 404 or 403, so
the app said to try again in a moment, or that the provider did not offer
the model. The provider's error code is now read before the status: no
credit, a spending limit and no access to the model each get their own
line.

Test connection only listed models, which such a key passes. Once a model
is chosen, the OpenAI and Anthropic tests send that model a few tokens
through the client real requests use and report what the provider said. A
reply cut off by the token cap counts as a pass. With no model chosen the
model list check is unchanged, and so is the OpenAI-compatible connector.

* Say so when a connection test fails, and keep an outage from switching AI off

The Test connection button showed "Connection works" even when the check
had failed; it now shows why the test failed. The AI probe in the
connection test is asked once with a time limit, and a busy or
unavailable provider falls back to the key check instead of marking the
connection failed.

* Show AI provider errors in the workshop's language

The lines that explain a failed AI request, no credit, a spending limit,
no access to the model, a rejected key and the rest, were English only
in every locale, both in toasts and on the connection page. They now
live under integrations.aiErrors in each locale and are read in the
language of the request. With no request to take a language from, such
as a background job, they stay in English. A provider's own message is
still passed through as it came.

messages/de/integrations.json

@@ -1085,5 +1085,17 @@
     "restoredBySupport": "Der Torqvoice-Support hat dies auf Ihre Anfrage hin durchgeführt. Angegebener Grund: {reason}",
     "noReason": "keiner angegeben",
     "restoredUndo": "Vorher wurde eine Sicherheitskopie der Werkstatt im Zustand direkt vor der Wiederherstellung erstellt, die 90 Tage lang aufbewahrt wird. Wenden Sie sich an den Support, um zu ihr zurückzukehren."
+  },
+  "aiErrors": {
+    "modelAccess": "Dieser API-Schlüssel hat keinen Zugriff auf das ausgewählte Modell. Wählen Sie unter Einstellungen → Integrationen ein anderes.",
+    "spendLimit": "Dieses OpenAI-Konto hat sein Ausgabenlimit erreicht. Prüfen Sie die Limits auf platform.openai.com.",
+    "noCredit": "Dieses OpenAI-Konto hat kein Guthaben mehr. Hinterlegen Sie eine Zahlungsmethode auf platform.openai.com.",
+    "keyRejected": "Der KI-Anbieter hat den API-Schlüssel abgelehnt. Prüfen Sie ihn unter Einstellungen → Integrationen.",
+    "noUsableCredit": "Der KI-Anbieter meldet kein nutzbares Guthaben für diesen API-Schlüssel.",
+    "modelUnavailable": "Der KI-Anbieter bietet das ausgewählte Modell nicht an. Wählen Sie unter Einstellungen → Integrationen ein anderes.",
+    "imageTooLarge": "Das Bild war für den KI-Anbieter zu groß.",
+    "rateLimited": "Der KI-Anbieter begrenzt die Anfragen dieses API-Schlüssels. Versuchen Sie es gleich noch einmal.",
+    "providerDown": "Der KI-Anbieter ist gerade nicht erreichbar. Versuchen Sie es gleich noch einmal.",
+    "unexpected": "Der KI-Anbieter hat eine unerwartete Antwort geliefert."
   }
 }

messages/en/integrations.json

@@ -1085,5 +1085,17 @@
     "restoredBySupport": "Torqvoice support did this at your request. Reason given: {reason}",
     "noReason": "none given",
     "restoredUndo": "A safety copy of the workshop as it was right before the restore was taken first and is kept for 90 days. Contact support to go back to it."
+  },
+  "aiErrors": {
+    "modelAccess": "This API key has no access to the selected model. Pick another in Settings → Integrations.",
+    "spendLimit": "This OpenAI account has reached its spending limit. Check the limits at platform.openai.com.",
+    "noCredit": "This OpenAI account has no credit left. Add billing at platform.openai.com.",
+    "keyRejected": "The AI provider rejected the API key. Check it in Settings → Integrations.",
+    "noUsableCredit": "The AI provider reports no usable credit for this API key.",
+    "modelUnavailable": "The AI provider does not offer the selected model. Pick another in Settings → Integrations.",
+    "imageTooLarge": "The image was too large for the AI provider.",
+    "rateLimited": "The AI provider is rate limiting this API key. Try again in a moment.",
+    "providerDown": "The AI provider is unavailable right now. Try again in a moment.",
+    "unexpected": "The AI provider returned an unexpected response."
   }
 }

messages/es/integrations.json

@@ -1085,5 +1085,17 @@
     "restoredBySupport": "El soporte de Torqvoice lo hizo a petición suya. Motivo indicado: {reason}",
     "noReason": "no se indicó ninguno",
     "restoredUndo": "Antes se hizo una copia de protección del taller tal como estaba justo antes de la restauración, que se conserva durante 90 días. Contacte con el soporte para volver a ella."
+  },
+  "aiErrors": {
+    "modelAccess": "Esta clave de API no tiene acceso al modelo seleccionado. Elija otro en Configuración → Integraciones.",
+    "spendLimit": "Esta cuenta de OpenAI ha alcanzado su límite de gasto. Revise los límites en platform.openai.com.",
+    "noCredit": "Esta cuenta de OpenAI no tiene saldo. Añada un método de pago en platform.openai.com.",
+    "keyRejected": "El proveedor de IA rechazó la clave de API. Revísela en Configuración → Integraciones.",
+    "noUsableCredit": "El proveedor de IA indica que esta clave de API no tiene saldo disponible.",
+    "modelUnavailable": "El proveedor de IA no ofrece el modelo seleccionado. Elija otro en Configuración → Integraciones.",
+    "imageTooLarge": "La imagen era demasiado grande para el proveedor de IA.",
+    "rateLimited": "El proveedor de IA está limitando las solicitudes de esta clave de API. Vuelva a intentarlo en un momento.",
+    "providerDown": "El proveedor de IA no está disponible ahora mismo. Vuelva a intentarlo en un momento.",
+    "unexpected": "El proveedor de IA devolvió una respuesta inesperada."
   }
 }

messages/fr/integrations.json

@@ -1085,5 +1085,17 @@
     "restoredBySupport": "L'assistance Torqvoice l'a fait à votre demande. Motif indiqué : {reason}",
     "noReason": "aucun",
     "restoredUndo": "Une copie de sécurité de l'atelier tel qu'il était juste avant la restauration a d'abord été réalisée et est conservée pendant 90 jours. Contactez l'assistance pour y revenir."
+  },
+  "aiErrors": {
+    "modelAccess": "Cette clé d'API n'a pas accès au modèle sélectionné. Choisissez-en un autre dans Paramètres → Intégrations.",
+    "spendLimit": "Ce compte OpenAI a atteint sa limite de dépenses. Vérifiez les limites sur platform.openai.com.",
+    "noCredit": "Ce compte OpenAI n'a plus de crédit. Ajoutez un moyen de paiement sur platform.openai.com.",
+    "keyRejected": "Le fournisseur d'IA a refusé la clé d'API. Vérifiez-la dans Paramètres → Intégrations.",
+    "noUsableCredit": "Le fournisseur d'IA n'indique aucun crédit utilisable pour cette clé d'API.",
+    "modelUnavailable": "Le fournisseur d'IA ne propose pas le modèle sélectionné. Choisissez-en un autre dans Paramètres → Intégrations.",
+    "imageTooLarge": "L'image était trop volumineuse pour le fournisseur d'IA.",
+    "rateLimited": "Le fournisseur d'IA limite les requêtes de cette clé d'API. Réessayez dans un instant.",
+    "providerDown": "Le fournisseur d'IA est indisponible pour le moment. Réessayez dans un instant.",
+    "unexpected": "Le fournisseur d'IA a renvoyé une réponse inattendue."
   }
 }

messages/it/integrations.json

@@ -1085,5 +1085,17 @@
     "restoredBySupport": "L'assistenza Torqvoice lo ha fatto su tua richiesta. Motivo indicato: {reason}",
     "noReason": "nessuno",
     "restoredUndo": "Prima è stata creata una copia di sicurezza dell'officina com'era subito prima del ripristino, che viene conservata per 90 giorni. Contatta l'assistenza per tornarci."
+  },
+  "aiErrors": {
+    "modelAccess": "Questa chiave API non ha accesso al modello selezionato. Scegline un altro in Impostazioni → Integrazioni.",
+    "spendLimit": "Questo account OpenAI ha raggiunto il limite di spesa. Controlla i limiti su platform.openai.com.",
+    "noCredit": "Questo account OpenAI non ha più credito. Aggiungi un metodo di pagamento su platform.openai.com.",
+    "keyRejected": "Il fornitore di IA ha rifiutato la chiave API. Controllala in Impostazioni → Integrazioni.",
+    "noUsableCredit": "Il fornitore di IA non segnala credito utilizzabile per questa chiave API.",
+    "modelUnavailable": "Il fornitore di IA non offre il modello selezionato. Scegline un altro in Impostazioni → Integrazioni.",
+    "imageTooLarge": "L'immagine era troppo grande per il fornitore di IA.",
+    "rateLimited": "Il fornitore di IA sta limitando le richieste di questa chiave API. Riprova tra un momento.",
+    "providerDown": "Il fornitore di IA non è disponibile al momento. Riprova tra un momento.",
+    "unexpected": "Il fornitore di IA ha restituito una risposta inattesa."
   }
 }

messages/lt/integrations.json

@@ -1085,5 +1085,17 @@
     "restoredBySupport": "Torqvoice pagalbos komanda tai atliko jūsų prašymu. Nurodyta priežastis: {reason}",
     "noReason": "nenurodyta",
     "restoredUndo": "Prieš atkuriant buvo sukurta dirbtuvės apsauginė kopija, kokia ji buvo prieš pat atkūrimą; ji saugoma 90 dienų. Norėdami prie jos grįžti, susisiekite su pagalbos komanda."
+  },
+  "aiErrors": {
+    "modelAccess": "Šis API raktas neturi prieigos prie pasirinkto modelio. Pasirinkite kitą skiltyje Nustatymai → Integracijos.",
+    "spendLimit": "Ši OpenAI paskyra pasiekė išlaidų limitą. Patikrinkite limitus platform.openai.com.",
+    "noCredit": "Šioje OpenAI paskyroje nebeliko kredito. Pridėkite mokėjimo būdą platform.openai.com.",
+    "keyRejected": "DI paslaugų teikėjas atmetė API raktą. Patikrinkite jį skiltyje Nustatymai → Integracijos.",
+    "noUsableCredit": "DI paslaugų teikėjas praneša, kad šiam API raktui nėra naudojamo kredito.",
+    "modelUnavailable": "DI paslaugų teikėjas nesiūlo pasirinkto modelio. Pasirinkite kitą skiltyje Nustatymai → Integracijos.",
+    "imageTooLarge": "Vaizdas buvo per didelis DI paslaugų teikėjui.",
+    "rateLimited": "DI paslaugų teikėjas riboja šio API rakto užklausas. Netrukus bandykite dar kartą.",
+    "providerDown": "DI paslaugų teikėjas šiuo metu nepasiekiamas. Netrukus bandykite dar kartą.",
+    "unexpected": "DI paslaugų teikėjas grąžino netikėtą atsakymą."
   }
 }

messages/nb/integrations.json

@@ -1085,5 +1085,17 @@
     "restoredBySupport": "Torqvoice-supporten gjorde dette etter forespørsel fra deg. Oppgitt grunn: {reason}",
     "noReason": "ingen oppgitt",
     "restoredUndo": "Først ble det tatt en kopi av verkstedet slik det var rett før gjenopprettingen, og den beholdes i 90 dager. Kontakt supporten for å gå tilbake til den."
+  },
+  "aiErrors": {
+    "modelAccess": "Denne API-nøkkelen har ikke tilgang til den valgte modellen. Velg en annen under Innstillinger → Integrasjoner.",
+    "spendLimit": "Denne OpenAI-kontoen har nådd forbruksgrensen sin. Sjekk grensene på platform.openai.com.",
+    "noCredit": "Denne OpenAI-kontoen har ikke mer kreditt. Legg til betaling på platform.openai.com.",
+    "keyRejected": "AI-leverandøren avviste API-nøkkelen. Sjekk den under Innstillinger → Integrasjoner.",
+    "noUsableCredit": "AI-leverandøren melder at denne API-nøkkelen ikke har kreditt som kan brukes.",
+    "modelUnavailable": "AI-leverandøren tilbyr ikke den valgte modellen. Velg en annen under Innstillinger → Integrasjoner.",
+    "imageTooLarge": "Bildet var for stort for AI-leverandøren.",
+    "rateLimited": "AI-leverandøren begrenser forespørslene fra denne API-nøkkelen. Prøv igjen om litt.",
+    "providerDown": "AI-leverandøren er utilgjengelig akkurat nå. Prøv igjen om litt.",
+    "unexpected": "AI-leverandøren ga et uventet svar."
   }
 }

messages/nl/integrations.json

@@ -1085,5 +1085,17 @@
     "restoredBySupport": "Torqvoice-support heeft dit op uw verzoek gedaan. Opgegeven reden: {reason}",
     "noReason": "geen opgegeven",
     "restoredUndo": "Eerst is een veiligheidskopie gemaakt van de werkplaats zoals die vlak voor het terugzetten was; die wordt 90 dagen bewaard. Neem contact op met support om ernaar terug te gaan."
+  },
+  "aiErrors": {
+    "modelAccess": "Deze API-sleutel heeft geen toegang tot het gekozen model. Kies een ander onder Instellingen → Integraties.",
+    "spendLimit": "Dit OpenAI-account heeft zijn bestedingslimiet bereikt. Controleer de limieten op platform.openai.com.",
+    "noCredit": "Dit OpenAI-account heeft geen tegoed meer. Voeg betaalgegevens toe op platform.openai.com.",
+    "keyRejected": "De AI-aanbieder heeft de API-sleutel geweigerd. Controleer die onder Instellingen → Integraties.",
+    "noUsableCredit": "De AI-aanbieder meldt geen bruikbaar tegoed voor deze API-sleutel.",
+    "modelUnavailable": "De AI-aanbieder biedt het gekozen model niet aan. Kies een ander onder Instellingen → Integraties.",
+    "imageTooLarge": "De afbeelding was te groot voor de AI-aanbieder.",
+    "rateLimited": "De AI-aanbieder beperkt de verzoeken van deze API-sleutel. Probeer het zo meteen opnieuw.",
+    "providerDown": "De AI-aanbieder is op dit moment niet beschikbaar. Probeer het zo meteen opnieuw.",
+    "unexpected": "De AI-aanbieder gaf een onverwacht antwoord."
   }
 }

messages/pl/integrations.json

@@ -1085,5 +1085,17 @@
     "restoredBySupport": "Zespół pomocy Torqvoice zrobił to na Twoją prośbę. Podany powód: {reason}",
     "noReason": "nie podano",
     "restoredUndo": "Najpierw utworzono kopię bezpieczeństwa warsztatu w stanie sprzed przywrócenia, która jest przechowywana przez 90 dni. Skontaktuj się z zespołem pomocy, aby do niej wrócić."
+  },
+  "aiErrors": {
+    "modelAccess": "Ten klucz API nie ma dostępu do wybranego modelu. Wybierz inny w Ustawienia → Integracje.",
+    "spendLimit": "To konto OpenAI osiągnęło limit wydatków. Sprawdź limity na platform.openai.com.",
+    "noCredit": "Na tym koncie OpenAI nie ma już środków. Dodaj płatność na platform.openai.com.",
+    "keyRejected": "Dostawca AI odrzucił klucz API. Sprawdź go w Ustawienia → Integracje.",
+    "noUsableCredit": "Dostawca AI zgłasza brak dostępnych środków dla tego klucza API.",
+    "modelUnavailable": "Dostawca AI nie oferuje wybranego modelu. Wybierz inny w Ustawienia → Integracje.",
+    "imageTooLarge": "Obraz był za duży dla dostawcy AI.",
+    "rateLimited": "Dostawca AI ogranicza liczbę zapytań z tego klucza API. Spróbuj ponownie za chwilę.",
+    "providerDown": "Dostawca AI jest teraz niedostępny. Spróbuj ponownie za chwilę.",
+    "unexpected": "Dostawca AI zwrócił nieoczekiwaną odpowiedź."
   }
 }

messages/pt-BR/integrations.json

@@ -1085,5 +1085,17 @@
     "restoredBySupport": "O suporte da Torqvoice fez isso a seu pedido. Motivo informado: {reason}",
     "noReason": "nenhum informado",
     "restoredUndo": "Antes, foi feita uma cópia de proteção da oficina como ela estava logo antes da restauração, que fica guardada por 90 dias. Fale com o suporte para voltar a ela."
+  },
+  "aiErrors": {
+    "modelAccess": "Esta chave de API não tem acesso ao modelo selecionado. Escolha outro em Configurações → Integrações.",
+    "spendLimit": "Esta conta da OpenAI atingiu o limite de gastos. Confira os limites em platform.openai.com.",
+    "noCredit": "Esta conta da OpenAI está sem crédito. Adicione um meio de pagamento em platform.openai.com.",
+    "keyRejected": "O provedor de IA recusou a chave de API. Confira-a em Configurações → Integrações.",
+    "noUsableCredit": "O provedor de IA informa que não há crédito disponível para esta chave de API.",
+    "modelUnavailable": "O provedor de IA não oferece o modelo selecionado. Escolha outro em Configurações → Integrações.",
+    "imageTooLarge": "A imagem era grande demais para o provedor de IA.",
+    "rateLimited": "O provedor de IA está limitando as solicitações desta chave de API. Tente novamente em instantes.",
+    "providerDown": "O provedor de IA está indisponível no momento. Tente novamente em instantes.",
+    "unexpected": "O provedor de IA retornou uma resposta inesperada."
   }
 }

messages/pt-PT/integrations.json

@@ -1085,5 +1085,17 @@
     "restoredBySupport": "O suporte do Torqvoice fez isto a seu pedido. Motivo indicado: {reason}",
     "noReason": "nenhum indicado",
     "restoredUndo": "Antes, foi feita uma cópia de proteção da oficina tal como estava imediatamente antes da reposição, que é guardada durante 90 dias. Contacte o suporte para voltar a ela."
+  },
+  "aiErrors": {
+    "modelAccess": "Esta chave de API não tem acesso ao modelo selecionado. Escolha outro em Definições → Integrações.",
+    "spendLimit": "Esta conta da OpenAI atingiu o limite de gastos. Verifique os limites em platform.openai.com.",
+    "noCredit": "Esta conta da OpenAI ficou sem crédito. Adicione um meio de pagamento em platform.openai.com.",
+    "keyRejected": "O fornecedor de IA recusou a chave de API. Verifique-a em Definições → Integrações.",
+    "noUsableCredit": "O fornecedor de IA indica que não há crédito utilizável para esta chave de API.",
+    "modelUnavailable": "O fornecedor de IA não disponibiliza o modelo selecionado. Escolha outro em Definições → Integrações.",
+    "imageTooLarge": "A imagem era demasiado grande para o fornecedor de IA.",
+    "rateLimited": "O fornecedor de IA está a limitar os pedidos desta chave de API. Tente novamente dentro de momentos.",
+    "providerDown": "O fornecedor de IA está indisponível neste momento. Tente novamente dentro de momentos.",
+    "unexpected": "O fornecedor de IA devolveu uma resposta inesperada."
   }
 }

messages/ru/integrations.json

@@ -1085,5 +1085,17 @@
     "restoredBySupport": "Служба поддержки Torqvoice сделала это по вашей просьбе. Указанная причина: {reason}",
     "noReason": "не указана",
     "restoredUndo": "Перед этим была создана страховочная копия мастерской в том виде, в каком она была непосредственно перед восстановлением; она хранится 90 дней. Чтобы вернуться к ней, обратитесь в службу поддержки."
+  },
+  "aiErrors": {
+    "modelAccess": "У этого API-ключа нет доступа к выбранной модели. Выберите другую в разделе Настройки → Интеграции.",
+    "spendLimit": "Этот аккаунт OpenAI достиг лимита расходов. Проверьте лимиты на platform.openai.com.",
+    "noCredit": "На этом аккаунте OpenAI закончились средства. Добавьте способ оплаты на platform.openai.com.",
+    "keyRejected": "Поставщик ИИ отклонил API-ключ. Проверьте его в разделе Настройки → Интеграции.",
+    "noUsableCredit": "Поставщик ИИ сообщает, что у этого API-ключа нет доступных средств.",
+    "modelUnavailable": "Поставщик ИИ не предлагает выбранную модель. Выберите другую в разделе Настройки → Интеграции.",
+    "imageTooLarge": "Изображение слишком большое для поставщика ИИ.",
+    "rateLimited": "Поставщик ИИ ограничивает запросы с этого API-ключа. Попробуйте ещё раз чуть позже.",
+    "providerDown": "Поставщик ИИ сейчас недоступен. Попробуйте ещё раз чуть позже.",
+    "unexpected": "Поставщик ИИ вернул неожиданный ответ."
   }
 }

messages/tr/integrations.json

@@ -1085,5 +1085,17 @@
     "restoredBySupport": "Bunu Torqvoice destek ekibi sizin isteğiniz üzerine yaptı. Belirtilen neden: {reason}",
     "noReason": "belirtilmedi",
     "restoredUndo": "Önce atölyenin geri yüklemeden hemen önceki hâlinin bir güvenlik kopyası alındı ve 90 gün saklanıyor. Ona geri dönmek için destek ekibiyle iletişime geçin."
+  },
+  "aiErrors": {
+    "modelAccess": "Bu API anahtarının seçilen modele erişimi yok. Ayarlar → Entegrasyonlar bölümünden başka bir model seçin.",
+    "spendLimit": "Bu OpenAI hesabı harcama sınırına ulaştı. Sınırları platform.openai.com adresinden kontrol edin.",
+    "noCredit": "Bu OpenAI hesabında kredi kalmadı. platform.openai.com adresinden ödeme bilgisi ekleyin.",
+    "keyRejected": "Yapay zekâ sağlayıcısı API anahtarını reddetti. Ayarlar → Entegrasyonlar bölümünden kontrol edin.",
+    "noUsableCredit": "Yapay zekâ sağlayıcısı bu API anahtarı için kullanılabilir kredi olmadığını bildiriyor.",
+    "modelUnavailable": "Yapay zekâ sağlayıcısı seçilen modeli sunmuyor. Ayarlar → Entegrasyonlar bölümünden başka bir model seçin.",
+    "imageTooLarge": "Görsel, yapay zekâ sağlayıcısı için çok büyüktü.",
+    "rateLimited": "Yapay zekâ sağlayıcısı bu API anahtarının isteklerini sınırlıyor. Birazdan tekrar deneyin.",
+    "providerDown": "Yapay zekâ sağlayıcısına şu anda ulaşılamıyor. Birazdan tekrar deneyin.",
+    "unexpected": "Yapay zekâ sağlayıcısı beklenmeyen bir yanıt döndürdü."
   }
 }

src/__tests__/app/integrations/connection-test-toast.test.tsx

@@ -0,0 +1,112 @@
+/**
+ * The Test connection button on a connector's page. The action answers
+ * `success: true` whenever the check ran, with what the check found in
+ * `data.ok`, so a key without credit used to be told "Connection works" in
+ * green. The toast now follows what the check found.
+ */
+import { fireEvent, render, screen, waitFor } from '@testing-library/react'
+import { beforeEach, describe, expect, it, vi } from 'vitest'
+import { ConnectionSettings } from '@/app/(authenticated)/settings/integrations/[connector]/connection-settings'
+import type { ConnectionView } from '@/features/integrations/Actions/integrationActions'
+import { testIntegration } from '@/features/integrations/Actions/integrationActions'
+import type { ActivityPage } from '@/features/integrations/Lib/types'
+import { manifest } from '@/integrations/openai/manifest'
+import { toast } from 'sonner'
+
+vi.mock('sonner', () => ({ toast: { success: vi.fn(), error: vi.fn() } }))
+vi.mock('next/navigation', () => ({
+  useRouter: () => ({ refresh: vi.fn(), replace: vi.fn(), push: vi.fn() }),
+  useSearchParams: () => new URLSearchParams(),
+}))
+vi.mock('@/components/confirm-dialog', () => ({ useConfirm: () => vi.fn() }))
+vi.mock('@/components/upgrade-gate', () => ({ handleGated: () => false }))
+vi.mock('@/features/integrations/Actions/integrationActions', () => ({
+  backfillIntegrationAccounting: vi.fn(),
+  backfillIntegrationCalendar: vi.fn(),
+  disconnectIntegration: vi.fn(),
+  // The model list on OpenAI's settings asks the vendor as the page opens.
+  getIntegrationRemoteOptions: vi.fn(async () => ({ success: true, data: [] })),
+  previewIntegrationRemoteOptions: vi.fn(),
+  previewIntegrationTest: vi.fn(),
+  retryIntegrationBatch: vi.fn(),
+  retryIntegrationJob: vi.fn(),
+  runIntegrationJob: vi.fn(),
+  saveIntegrationCredentials: vi.fn(),
+  sendIntegrationTestMessage: vi.fn(),
+  testIntegration: vi.fn(),
+  updateIntegrationSettings: vi.fn(),
+}))
+vi.mock('@/features/integrations/Hooks/useIntegrationActivity', () => ({
+  useIntegrationActivity: (_org: string, _id: string, activity: ActivityPage) => ({
+    data: { pages: [activity] },
+    fetchNextPage: vi.fn(),
+    hasNextPage: false,
+    isFetchingNextPage: false,
+    isFetchNextPageError: false,
+  }),
+}))
+
+const view: ConnectionView = {
+  manifest,
+  connection: {
+    id: 'conn-1',
+    status: 'active',
+    externalAccountName: null,
+    externalAccountId: null,
+    settings: {},
+    lastHealthAt: null,
+    lastSyncAt: null,
+    lastError: null,
+    createdAt: '2026-11-01T10:00:00.000Z',
+    tenantClientId: null,
+  },
+  platformApp: false,
+  enabled: true,
+  isCloud: false,
+  webhookUrl: null,
+  inspectionSync: null,
+  backup: null,
+  inbound: null,
+  awaitingKeys: [],
+  supersedes: null,
+  redirectUri: 'http://localhost:3000/api/integrations/openai/callback',
+}
+
+const activity: ActivityPage = { items: [], logs: [], nextCursor: null }
+
+async function pressTest(answer: Awaited<ReturnType<typeof testIntegration>>) {
+  vi.mocked(testIntegration).mockResolvedValue(answer)
+  render(<ConnectionSettings organizationId="org-1" view={view} activity={activity} />)
+  fireEvent.click(screen.getByRole('button', { name: 'Test connection' }))
+  await waitFor(() => expect(testIntegration).toHaveBeenCalledWith('openai'))
+  // The button comes back once the test has finished.
+  await waitFor(() => expect(screen.getByRole('button', { name: 'Test connection' })).toBeEnabled())
+}
+
+describe('Test connection on a connected OpenAI page', () => {
+  beforeEach(() => {
+    vi.clearAllMocks()
+  })
+
+  it('shows why a failed test failed, and never says the connection works', async () => {
+    const message = 'This OpenAI account has no credit left. Add billing at platform.openai.com.'
+    await pressTest({ success: true, data: { ok: false, message } })
+
+    expect(toast.error).toHaveBeenCalledWith(message)
+    expect(toast.success).not.toHaveBeenCalledWith('Connection works')
+  })
+
+  it('says the test failed when the check gives no reason', async () => {
+    await pressTest({ success: true, data: { ok: false } })
+
+    expect(toast.error).toHaveBeenCalledWith('The test failed')
+    expect(toast.success).not.toHaveBeenCalledWith('Connection works')
+  })
+
+  it('says the connection works when the check passed', async () => {
+    await pressTest({ success: true, data: { ok: true } })
+
+    expect(toast.success).toHaveBeenCalledWith('Connection works')
+    expect(toast.error).not.toHaveBeenCalled()
+  })
+})

src/__tests__/features/integrations/ai-connection-test.test.ts

@@ -0,0 +1,290 @@
+// @vitest-environment node
+/**
+ * The "Test connection" button on the OpenAI and Anthropic connectors.
+ *
+ * Listing models proves a key exists. A workshop's OpenAI key did that, the
+ * test passed, and every request afterwards failed because the account had
+ * no credit. Once a model is chosen the test has to ask that model for
+ * something, and say what the provider said when it will not answer.
+ *
+ * The provider is played at the network: the SDK is the real one, so what
+ * these assert is what a workshop reads for a given answer from the vendor.
+ */
+
+import { afterEach, describe, expect, it, vi } from 'vitest'
+import type { ConnectorContext } from '@/features/integrations/Lib/types'
+import { connector as anthropic } from '@/integrations/anthropic/server'
+import { connector as openai } from '@/integrations/openai/server'
+
+vi.mock('@/lib/db', () => ({ db: {} }))
+
+type Provider = (url: string, init?: RequestInit) => Response
+
+/** A saved connection; `listing` is what the model list answers through the connector's own client. */
+function connection(
+  connectorId: string,
+  settings: Record<string, unknown>,
+  listing: Response = json(200, { data: [{ id: 'gpt-4.1' }] })
+): ConnectorContext {
+  const fetch = vi.fn(async (_url: string, _init?: RequestInit) => listing)
+  return {
+    connection: {
+      id: 'c1',
+      organizationId: 'org',
+      connectorId,
+      settings,
+      state: {},
+      externalAccountId: null,
+    },
+    credentials: { apiKey: 'sk-test' },
+    http: {
+      fetch,
+      async json<T>(url: string, init?: RequestInit): Promise<T> {
+        return (await (await fetch(url, init)).json()) as T
+      },
+    },
+    links: { get: vi.fn(), set: vi.fn(), remove: vi.fn(), remoteIds: vi.fn(), byRemoteId: vi.fn() },
+    log: vi.fn(async () => undefined),
+    saveState: vi.fn(),
+    timezone: 'Europe/Paris',
+    appUrl: 'https://app.test',
+  }
+}
+
+function json(status: number, body: unknown, headers: Record<string, string> = {}): Response {
+  return new Response(JSON.stringify(body), {
+    status,
+    headers: { 'content-type': 'application/json', ...headers },
+  })
+}
+
+/** Stand in for the vendor's API, which completions reach through the SDK. */
+function provider(answer: Provider) {
+  const calls: { url: string; body: Record<string, unknown> }[] = []
+  vi.stubGlobal(
+    'fetch',
+    vi.fn(async (input: string | URL | Request, init?: RequestInit) => {
+      const url = input instanceof Request ? input.url : String(input)
+      calls.push({ url, body: JSON.parse(typeof init?.body === 'string' ? init.body : '{}') })
+      return answer(url, init)
+    })
+  )
+  return calls
+}
+
+/** A completion that stopped at the token cap: thinking used it up, nothing was written. */
+const cutOff = (model: string) =>
+  json(200, {
+    id: 'chatcmpl-1',
+    object: 'chat.completion',
+    created: 1,
+    model,
+    choices: [{ index: 0, message: { role: 'assistant', content: '' }, finish_reason: 'length' }],
+    usage: { prompt_tokens: 9, completion_tokens: 16, total_tokens: 25 },
+  })
+
+/** A refusal in OpenAI's shape. The SDK retries a 429, so it is told not to wait between tries. */
+const refusal = (status: number, error: { message: string; type: string; code: string | null }) =>
+  json(status, { error: { param: null, ...error } }, { 'retry-after-ms': '1' })
+
+describe('AI connection test', () => {
+  afterEach(() => {
+    vi.unstubAllGlobals()
+    vi.restoreAllMocks()
+  })
+
+  describe('OpenAI', () => {
+    it('fails with the credit message when the account cannot pay for a completion', async () => {
+      vi.spyOn(console, 'error').mockImplementation(() => undefined)
+      provider(() =>
+        refusal(429, {
+          message: 'You exceeded your current quota, please check your plan and billing details.',
+          type: 'insufficient_quota',
+          code: 'insufficient_quota',
+        })
+      )
+
+      await expect(openai.test(connection('openai', { model: 'gpt-4.1' }))).resolves.toEqual({
+        ok: false,
+        message: 'This OpenAI account has no credit left. Add billing at platform.openai.com.',
+      })
+    })
+
+    it('fails when the key may not use the chosen model, and says so', async () => {
+      vi.spyOn(console, 'error').mockImplementation(() => undefined)
+      provider(() =>
+        refusal(404, {
+          message: 'The model `gpt-4.1-nano` does not exist or you do not have access to it.',
+          type: 'invalid_request_error',
+          code: 'model_not_found',
+        })
+      )
+
+      await expect(openai.test(connection('openai', { model: 'gpt-4.1-nano' }))).resolves.toEqual({
+        ok: false,
+        message:
+          'This API key has no access to the selected model. Pick another in Settings → Integrations.',
+      })
+    })
+
+    it('passes on a reply the token cap cut off', async () => {
+      const calls = provider(() => cutOff('gpt-5-mini'))
+
+      await expect(openai.test(connection('openai', { model: 'gpt-5-mini' }))).resolves.toEqual({
+        ok: true,
+      })
+      // The model the workshop picked is the one that was asked, and without
+      // a temperature, which the reasoning models answer with a 400.
+      expect(calls).toHaveLength(1)
+      expect(calls[0].url).toBe('https://api.openai.com/v1/chat/completions')
+      expect(calls[0].body.model).toBe('gpt-5-mini')
+      expect(calls[0].body).not.toHaveProperty('temperature')
+    })
+
+    it('passes when the provider reports the cap as an error instead of a short reply', async () => {
+      provider(() =>
+        refusal(400, {
+          message:
+            'Could not finish the message because max_tokens or model output limit was reached. Please try again with higher max_tokens.',
+          type: 'invalid_request_error',
+          code: null,
+        })
+      )
+
+      await expect(openai.test(connection('openai', { model: 'gpt-5-nano' }))).resolves.toEqual({
+        ok: true,
+      })
+    })
+
+    it('asks once, so a refusal is not sat through three times', async () => {
+      vi.spyOn(console, 'error').mockImplementation(() => undefined)
+      const calls = provider(() =>
+        refusal(429, {
+          message: 'You exceeded your current quota, please check your plan and billing details.',
+          type: 'insufficient_quota',
+          code: 'insufficient_quota',
+        })
+      )
+
+      await openai.test(connection('openai', { model: 'gpt-4.1' }))
+
+      expect(calls).toHaveLength(1)
+    })
+
+    it('does not fail a working connection because the provider was busy', async () => {
+      // A failed test switches the connection off, and a rate limit says
+      // nothing about the key: the key check decides instead.
+      vi.spyOn(console, 'error').mockImplementation(() => undefined)
+      provider(() =>
+        refusal(429, {
+          message: 'Rate limit reached for requests.',
+          type: 'requests',
+          code: 'rate_limit_exceeded',
+        })
+      )
+      const ctx = connection('openai', { model: 'gpt-4.1' })
+
+      await expect(openai.test(ctx)).resolves.toEqual({ ok: true })
+      expect(ctx.http.fetch).toHaveBeenCalledWith(
+        'https://api.openai.com/v1/models',
+        expect.objectContaining({ headers: { Authorization: 'Bearer sk-test' } })
+      )
+    })
+
+    it('nor because the provider was down', async () => {
+      vi.spyOn(console, 'error').mockImplementation(() => undefined)
+      provider(() => json(503, { error: { message: 'Service unavailable', type: 'server_error' } }))
+
+      await expect(openai.test(connection('openai', { model: 'gpt-4.1' }))).resolves.toEqual({
+        ok: true,
+      })
+    })
+
+    it('still fails when the provider is busy and the key check fails too', async () => {
+      vi.spyOn(console, 'error').mockImplementation(() => undefined)
+      provider(() => json(503, { error: { message: 'Service unavailable', type: 'server_error' } }))
+
+      await expect(
+        openai.test(connection('openai', { model: 'gpt-4.1' }, json(401, { error: {} })))
+      ).resolves.toEqual({ ok: false, message: 'OpenAI rejected the key' })
+    })
+
+    it('lists models, and asks for no completion, while no model is chosen', async () => {
+      const calls = provider(() => cutOff('gpt-4.1'))
+      const ctx = connection('openai', {})
+
+      await expect(openai.test(ctx)).resolves.toEqual({ ok: true })
+      expect(ctx.http.fetch).toHaveBeenCalledWith(
+        'https://api.openai.com/v1/models',
+        expect.objectContaining({ headers: { Authorization: 'Bearer sk-test' } })
+      )
+      expect(calls).toHaveLength(0)
+    })
+
+    it('still names a rejected key before a model is chosen', async () => {
+      provider(() => cutOff('gpt-4.1'))
+
+      await expect(
+        openai.test(connection('openai', { model: '  ' }, json(401, { error: {} })))
+      ).resolves.toEqual({ ok: false, message: 'OpenAI rejected the key' })
+    })
+  })
+
+  describe('Anthropic', () => {
+    it('fails with what the provider refused the completion for', async () => {
+      vi.spyOn(console, 'error').mockImplementation(() => undefined)
+      provider(() =>
+        refusal(402, {
+          message: 'There is an issue with your billing or payment information.',
+          type: 'billing_error',
+          code: null,
+        })
+      )
+
+      await expect(
+        anthropic.test(connection('anthropic', { model: 'claude-sonnet-4-6' }))
+      ).resolves.toEqual({
+        ok: false,
+        message: 'The AI provider reports no usable credit for this API key.',
+      })
+    })
+
+    it('passes on a reply the token cap cut off', async () => {
+      const calls = provider(() => cutOff('claude-opus-5-5'))
+
+      await expect(
+        anthropic.test(connection('anthropic', { model: 'claude-opus-5-5' }))
+      ).resolves.toEqual({ ok: true })
+      expect(calls).toHaveLength(1)
+      expect(calls[0].url).toBe('https://api.anthropic.com/v1/chat/completions')
+      expect(calls[0].body.model).toBe('claude-opus-5-5')
+      expect(calls[0].body).not.toHaveProperty('temperature')
+    })
+
+    it('does not fail a working connection because the provider was overloaded', async () => {
+      vi.spyOn(console, 'error').mockImplementation(() => undefined)
+      provider(() => json(529, { error: { message: 'Overloaded', type: 'overloaded_error' } }))
+      const ctx = connection('anthropic', { model: 'claude-sonnet-4-6' })
+
+      await expect(anthropic.test(ctx)).resolves.toEqual({ ok: true })
+      expect(ctx.http.fetch).toHaveBeenCalledWith(
+        'https://api.anthropic.com/v1/models?limit=1',
+        expect.anything()
+      )
+    })
+
+    it('lists models, and asks for no completion, while no model is chosen', async () => {
+      const calls = provider(() => cutOff('claude-opus-5-5'))
+      const ctx = connection('anthropic', {})
+
+      await expect(anthropic.test(ctx)).resolves.toEqual({ ok: true })
+      expect(ctx.http.fetch).toHaveBeenCalledWith(
+        'https://api.anthropic.com/v1/models?limit=1',
+        expect.objectContaining({
+          headers: expect.objectContaining({ 'x-api-key': 'sk-test' }),
+        })
+      )
+      expect(calls).toHaveLength(0)
+    })
+  })
+})

src/__tests__/lib/ai-error-message.test.ts

@@ -0,0 +1,67 @@
+/**
+ * The AI failure a workshop reads in a toast or on the connection page is in
+ * the language they use the app in, not always English.
+ */
+
+import { beforeEach, describe, expect, it, vi } from 'vitest'
+import OpenAI from 'openai'
+import nb from '../../../messages/nb/integrations.json'
+
+const { getTranslations } = vi.hoisted(() => ({ getTranslations: vi.fn() }))
+vi.mock('next-intl/server', () => ({ getTranslations }))
+
+import { aiErrorDescriber } from '@/lib/ai-error-message'
+
+function apiError(status: number, message: string, body?: { code?: string; type?: string }) {
+  return new OpenAI.APIError(status, body && { message, ...body }, message, undefined)
+}
+
+/** A translator over the Norwegian file, the way next-intl resolves one. */
+function norwegian(ns: string) {
+  expect(ns).toBe('integrations.aiErrors')
+  return (key: keyof typeof nb.aiErrors) => nb.aiErrors[key]
+}
+
+describe('aiErrorDescriber', () => {
+  beforeEach(() => {
+    getTranslations.mockReset()
+  })
+
+  it('tells an account without credit in the reader’s language', async () => {
+    getTranslations.mockImplementation(async (ns: string) => norwegian(ns))
+    const describeError = await aiErrorDescriber()
+    const message = describeError(
+      apiError(429, 'You exceeded your current quota', {
+        code: 'insufficient_quota',
+        type: 'insufficient_quota',
+      })
+    )
+    expect(message).toBe(nb.aiErrors.noCredit)
+    expect(message).toContain('kreditt')
+  })
+
+  it('translates a spending limit and a model the key may not use', async () => {
+    getTranslations.mockImplementation(async (ns: string) => norwegian(ns))
+    const describeError = await aiErrorDescriber()
+    expect(describeError(apiError(429, 'limit', { code: 'project_spend_limit_exceeded' }))).toBe(
+      nb.aiErrors.spendLimit
+    )
+    expect(describeError(apiError(403, 'no access', { code: 'model_not_found' }))).toBe(
+      nb.aiErrors.modelAccess
+    )
+  })
+
+  it('passes the provider’s own words through untranslated', async () => {
+    getTranslations.mockImplementation(async (ns: string) => norwegian(ns))
+    const describeError = await aiErrorDescriber()
+    expect(describeError(new Error('Context length exceeded'))).toBe('Context length exceeded')
+  })
+
+  it('falls back to English where there is no request language', async () => {
+    getTranslations.mockRejectedValue(new Error('no request scope'))
+    const describeError = await aiErrorDescriber()
+    expect(describeError(apiError(429, 'quota', { code: 'insufficient_quota' }))).toContain(
+      'no credit left'
+    )
+  })
+})

src/__tests__/lib/ai-error.test.ts

@@ -11,8 +11,9 @@ import { describe, it, expect } from 'vitest'
 import OpenAI from 'openai'
 import { describeAiError } from '@/lib/ai-error'
 
-function apiError(status: number, message: string) {
-  return new OpenAI.APIError(status, undefined, message, undefined)
+/** A failed request as the SDK raises it; `body` is the provider's own `error` object. */
+function apiError(status: number, message: string, body?: { code?: string; type?: string }) {
+  return new OpenAI.APIError(status, body && { message, ...body }, message, undefined)
 }
 
 describe('describeAiError', () => {
@@ -49,6 +50,80 @@ describe('describeAiError', () => {
     expect(describeAiError(apiError(429, 'slow down'))).toContain('rate limit')
   })
 
+  it('still calls a real rate limit a rate limit when the provider names it', () => {
+    const message = describeAiError(
+      apiError(429, 'Rate limit reached for gpt-4.1 on requests per min (RPM): Limit 3, Used 3.', {
+        code: 'rate_limit_exceeded',
+        type: 'requests',
+      })
+    )
+    expect(message).toBe('The AI provider is rate limiting this API key. Try again in a moment.')
+  })
+
+  // OpenAI answers an account with no paid credit with the same 429 as a
+  // rate limit. A workshop was told to try again in a moment, for a day.
+  it('tells an account without credit to add billing, not to wait', () => {
+    const noCredit = 'This OpenAI account has no credit left. Add billing at platform.openai.com.'
+    const quota = 'You exceeded your current quota, please check your plan and billing details.'
+
+    expect(
+      describeAiError(
+        apiError(429, quota, { code: 'insufficient_quota', type: 'insufficient_quota' })
+      )
+    ).toBe(noCredit)
+    // The code OpenAI documents today, under the same broader type.
+    expect(
+      describeAiError(
+        apiError(429, quota, { code: 'credit_balance_exhausted', type: 'insufficient_quota' })
+      )
+    ).toBe(noCredit)
+  })
+
+  it('names a spending limit when that is what stopped the request', () => {
+    for (const code of [
+      'organization_spend_limit_exceeded',
+      'project_spend_limit_exceeded',
+      'organization_usage_limit_exceeded',
+    ]) {
+      const message = describeAiError(
+        apiError(429, 'Spend limit reached.', { code, type: 'insufficient_quota' })
+      )
+      expect(message, code).toContain('spending limit')
+      expect(message, code).toContain('platform.openai.com')
+      expect(message, code).not.toContain('Try again')
+    }
+  })
+
+  // The model exists and the key is good; the key's project is not allowed
+  // to use it. OpenAI sends this as 404 or as 403, with the same code.
+  it('says the key has no access to the model, whichever status carries it', () => {
+    const noAccess =
+      'This API key has no access to the selected model. Pick another in Settings → Integrations.'
+
+    expect(
+      describeAiError(
+        apiError(404, 'The model `gpt-4.1` does not exist or you do not have access to it.', {
+          code: 'model_not_found',
+          type: 'invalid_request_error',
+        })
+      )
+    ).toBe(noAccess)
+    expect(
+      describeAiError(
+        apiError(403, 'Project `proj_1` does not have access to model `gpt-4.1`', {
+          code: 'model_not_found',
+          type: 'invalid_request_error',
+        })
+      )
+    ).toBe(noAccess)
+  })
+
+  it('keeps the plain answer for a 404 that carries no code', () => {
+    expect(describeAiError(apiError(404, 'Not found'))).toContain(
+      'does not offer the selected model'
+    )
+  })
+
   it('falls back when there is no message at all', () => {
     expect(describeAiError(new Error(''))).toBe('The AI provider returned an unexpected response.')
   })

src/app/(authenticated)/settings/integrations/[connector]/connection-settings.tsx

@@ -585,7 +585,17 @@ export function ConnectionSettings({
               variant="outline"
               size="sm"
               onClick={() =>
-                run('test', () => testIntegration(manifest.id), t('connection.testPassed'))
+                run('test', async () => {
+                  const res = await testIntegration(manifest.id)
+                  // The action succeeds whenever the check ran. What the check
+                  // found is in the data, and a failed one used to be told
+                  // "Connection works" in green above the red banner.
+                  if (res.success && res.data) {
+                    if (res.data.ok) toast.success(t('connection.testPassed'))
+                    else toast.error(res.data.message || t('connection.testFailed'))
+                  }
+                  return res
+                })
               }
               disabled={busy !== null}
             >

src/app/api/protected/ai/chat/route.ts

@@ -14,7 +14,7 @@ import {
 import { getFeatures } from '@/lib/features'
 import { db } from '@/lib/db'
 import { completionTuning, createClient, getAiConfig } from '@/lib/ai'
-import { describeAiError } from '@/lib/ai-error'
+import { aiErrorDescriber } from '@/lib/ai-error-message'
 import { localeNames, type Locale } from '@/i18n/config'
 import { SETTING_KEYS } from '@/features/settings/Schema/settingsSchema'
 import { workshopTimeZone } from '@/lib/workshop-timezone'
@@ -166,6 +166,8 @@ export async function POST(request: NextRequest) {
     { role: 'user', content: message },
   ]
 
+  // Fetched now: the stream below can fail after this request has returned.
+  const describeAiError = await aiErrorDescriber()
   const client = createClient(config)
   let stream: AsyncIterable<OpenAI.ChatCompletionChunk>
   try {

src/app/api/protected/ai/transcribe/route.ts

@@ -5,7 +5,7 @@ import { dictationAccess } from '@/features/ai/Lib/dictationAccess'
 import { rateLimit } from '@/lib/rate-limit'
 import { createClient } from '@/lib/ai'
 import { speechSetup } from '@/features/integrations/Lib/speech'
-import { describeAiError } from '@/lib/ai-error'
+import { aiErrorDescriber } from '@/lib/ai-error-message'
 import {
   MAX_TRANSCRIPTION_BYTES,
   transcriptionLanguage,
@@ -92,6 +92,6 @@ export async function POST(request: NextRequest) {
     })
     return NextResponse.json({ text: result.text.trim() })
   } catch (err) {
-    return json(502, describeAiError(err))
+    return json(502, (await aiErrorDescriber())(err))
   }
 }

src/features/import/Actions/importActions.ts

@@ -8,7 +8,7 @@ import { PermissionAction, PermissionSubject } from '@/lib/permissions'
 import { FeatureGatedError, getFeatures } from '@/lib/features'
 import { countCustomersTowardLimit } from '@/lib/customer-limit'
 import { completionTuning, createClient, getAiConfig } from '@/lib/ai'
-import { describeAiError } from '@/lib/ai-error'
+import { aiErrorDescriber } from '@/lib/ai-error-message'
 import { demoGuard } from '@/lib/demo'
 import { type ImportEntity, fieldsFor } from '../Lib/fields'
 import { permissionsFor } from '../Lib/permissions'
@@ -936,7 +936,7 @@ export async function suggestMappingWithAi(token: string) {
         })
         content = response.choices[0]?.message?.content ?? ''
       } catch (error) {
-        throw new Error(describeAiError(error))
+        throw new Error((await aiErrorDescriber())(error))
       }
 
       const json = content.match(/\{[\s\S]*\}/)?.[0]

src/integrations/ai/models.ts

@@ -20,6 +20,12 @@ export function apiKeyOf(ctx: ConnectorContext): string {
   return typeof key === 'string' ? key.trim() : ''
 }
 
+/** The model picked on the connection page; empty until the workshop has chosen one. */
+export function chosenModelOf(ctx: ConnectorContext): string {
+  const model = ctx.connection.settings.model
+  return typeof model === 'string' ? model.trim() : ''
+}
+
 /** Bearer auth, or nothing at all: a local server often has no key to check. */
 export function openAiHeaders(apiKey: string): Record<string, string> {
   return apiKey ? { Authorization: `Bearer ${apiKey}` } : {}

src/integrations/anthropic/server.ts

@@ -1,11 +1,23 @@
 import type { ConnectorServer } from '@/features/integrations/Lib/types'
-import { ANTHROPIC_BASE, anthropicHeaders, apiKeyOf, listAnthropicModels } from '../ai/models'
+import { probeAiModel } from '@/lib/ai'
+import {
+  ANTHROPIC_BASE,
+  anthropicHeaders,
+  apiKeyOf,
+  chosenModelOf,
+  listAnthropicModels,
+} from '../ai/models'
 import { manifest } from './manifest'
 
 /**
  * Anthropic authenticates with x-api-key rather than a bearer token, and
  * wants its API version on every request. Listing models proves the key
  * without spending tokens.
+ *
+ * As with OpenAI, a key that lists models can still be refused every
+ * completion, for want of credit or of access to the chosen model. Once a
+ * model is chosen the test asks it for a few tokens, the way a real request
+ * would. It only runs when someone connects or presses Test, never on a timer.
  */
 export const connector: ConnectorServer = {
   manifest,
@@ -13,6 +25,13 @@ export const connector: ConnectorServer = {
   async test(ctx) {
     const apiKey = apiKeyOf(ctx)
     if (!apiKey) return { ok: false, message: 'Anthropic: an API key is required' }
+    const model = chosenModelOf(ctx)
+    if (model) {
+      const probe = await probeAiModel({ provider: 'anthropic', apiKey, model })
+      // A provider that was busy or down says nothing about the key, and a
+      // failed test turns the connection off: fall through to the key check.
+      if (!probe.inconclusive) return { ok: probe.ok, message: probe.message }
+    }
     const res = await ctx.http.fetch(`${ANTHROPIC_BASE}/models?limit=1`, {
       headers: anthropicHeaders(apiKey),
     })

src/integrations/openai/server.ts

@@ -1,11 +1,18 @@
 import type { ConnectorServer } from '@/features/integrations/Lib/types'
-import { OPENAI_BASE, apiKeyOf, listOpenAiModels, openAiHeaders } from '../ai/models'
+import { probeAiModel } from '@/lib/ai'
+import { OPENAI_BASE, apiKeyOf, chosenModelOf, listOpenAiModels, openAiHeaders } from '../ai/models'
 import { manifest } from './manifest'
 
 /**
  * OpenAI needs nothing set up remotely: the key either reaches the account or
  * it does not. Listing models is the cheapest request that proves it, and it
  * is the same call the model dropdown makes.
+ *
+ * It does not prove the account can pay or that the key may use the model the
+ * workshop went on to pick: a key on an account without credit lists models
+ * and then fails every request. So once a model is chosen the test asks that
+ * model for a few tokens instead. It only runs when someone connects or
+ * presses Test, never on a timer.
  */
 export const connector: ConnectorServer = {
   manifest,
@@ -13,6 +20,13 @@ export const connector: ConnectorServer = {
   async test(ctx) {
     const apiKey = apiKeyOf(ctx)
     if (!apiKey) return { ok: false, message: 'OpenAI: an API key is required' }
+    const model = chosenModelOf(ctx)
+    if (model) {
+      const probe = await probeAiModel({ provider: 'openai', apiKey, model })
+      // A provider that was busy or down says nothing about the key, and a
+      // failed test turns the connection off: fall through to the key check.
+      if (!probe.inconclusive) return { ok: probe.ok, message: probe.message }
+    }
     const res = await ctx.http.fetch(`${OPENAI_BASE}/models`, { headers: openAiHeaders(apiKey) })
     if (res.status === 401 || res.status === 403)
       return { ok: false, message: 'OpenAI rejected the key' }

src/lib/ai-error-message.ts

@@ -0,0 +1,19 @@
+import { getTranslations } from 'next-intl/server'
+import { type AiErrorKind, describeAiError } from '@/lib/ai-error'
+
+/**
+ * `describeAiError` in the language of whoever made the request, for a line
+ * that reaches a toast or the connection page. The translator is fetched
+ * once, up front, so a stream that fails later still has it.
+ *
+ * Where there is no request to take a language from, such as a background
+ * job, the line stays in English rather than the failure being lost.
+ */
+export async function aiErrorDescriber(): Promise<(error: unknown) => string> {
+  try {
+    const t = await getTranslations('integrations.aiErrors')
+    return (error) => describeAiError(error, (kind: AiErrorKind) => t(kind))
+  } catch {
+    return (error) => describeAiError(error)
+  }
+}

src/lib/ai-error.ts

@@ -1,40 +1,98 @@
 import OpenAI from 'openai'
+import en from '../../messages/en/integrations.json'
 
 /**
- * Turns whatever a provider threw into one short line fit for a toast.
- *
- * A rejected key rarely comes back as tidy JSON: a gateway in front of the
- * provider answers with an HTML error page, and the SDK puts that entire page
- * in `error.message`. Users were seeing raw markup in the toast.
+ * What OpenAI puts in `error.code` when the account cannot pay. It answers
+ * these with HTTP 429, the status of a real rate limit, so the status alone
+ * told a workshop with an empty balance to "try again in a moment".
+ * `insufficient_quota` is the older spelling and is still the `error.type`
+ * of all of them.
+ */
+const NO_CREDIT_CODES = new Set(['insufficient_quota', 'credit_balance_exhausted'])
+const SPEND_LIMIT_CODES = new Set([
+  'organization_spend_limit_exceeded',
+  'project_spend_limit_exceeded',
+  'organization_usage_limit_exceeded',
+])
+
+/**
+ * Whether the provider refused because of the account's money rather than
+ * its traffic: no credit, or a spending limit reached. Both travel as 429,
+ * and unlike a real rate limit neither goes away by trying again.
+ */
+export function isAccountLimitError(error: unknown): boolean {
+  if (!(error instanceof OpenAI.APIError)) return false
+  const code = typeof error.code === 'string' ? error.code : ''
+  return (
+    NO_CREDIT_CODES.has(code) || SPEND_LIMIT_CODES.has(code) || error.type === 'insufficient_quota'
+  )
+}
+
+/**
+ * The failures a workshop can act on, each with a sentence under
+ * `integrations.aiErrors` in every locale. Anything else is the provider's
+ * own words.
+ */
+export type AiErrorKind = keyof typeof en.aiErrors
+
+/**
+ * Which known failure this is, or null when only the provider's own message
+ * can say. `unexpected` covers a reply with nothing a workshop can read.
  */
-export function describeAiError(error: unknown): string {
-  const generic = 'The AI provider returned an unexpected response.'
-  const status = error instanceof OpenAI.APIError ? error.status : undefined
+export function aiErrorKind(error: unknown): AiErrorKind | null {
+  const apiError = error instanceof OpenAI.APIError ? error : undefined
+  const status = apiError?.status
+  const code = typeof apiError?.code === 'string' ? apiError.code : ''
+
+  // The provider's own code says more than the status it travels under: a
+  // key that may not use a model comes back as 404 or as 403, and an account
+  // without credit as 429.
+  if (code === 'model_not_found') return 'modelAccess'
+  if (SPEND_LIMIT_CODES.has(code)) return 'spendLimit'
+  if (NO_CREDIT_CODES.has(code) || apiError?.type === 'insufficient_quota') return 'noCredit'
 
   switch (status) {
     case 401:
     case 403:
-      return 'The AI provider rejected the API key. Check it in Settings → Integrations.'
+      return 'keyRejected'
     case 402:
-      return 'The AI provider reports no usable credit for this API key.'
+      return 'noUsableCredit'
     case 404:
-      return 'The AI provider does not offer the selected model. Pick another in Settings → Integrations.'
+      return 'modelUnavailable'
     case 413:
-      return 'The image was too large for the AI provider.'
+      return 'imageTooLarge'
     case 429:
-      return 'The AI provider is rate limiting this API key. Try again in a moment.'
-  }
-  if (status && status >= 500) {
-    return 'The AI provider is unavailable right now. Try again in a moment.'
+      return 'rateLimited'
   }
-
-  const raw = error instanceof Error ? error.message : String(error ?? '')
+  if (status && status >= 500) return 'providerDown'
 
   // An HTML body says nothing a workshop can act on, and stripping the tags
   // just leaves the debris of a status page.
-  if (/<\s*(!doctype|html|head|body|div|p|span|title)\b/i.test(raw)) return generic
+  const raw = rawMessage(error)
+  if (!raw || /<\s*(!doctype|html|head|body|div|p|span|title)\b/i.test(raw)) return 'unexpected'
+  return null
+}
+
+function rawMessage(error: unknown): string {
+  const raw = error instanceof Error ? error.message : String(error ?? '')
+  return raw.replace(/\s+/g, ' ').trim()
+}
 
-  const text = raw.replace(/\s+/g, ' ').trim()
-  if (!text) return generic
+/**
+ * Turns whatever a provider threw into one short line fit for a toast, in
+ * English. Where a request has a language, `aiErrorDescriber` in
+ * `@/lib/ai-error-message` gives the same line in it.
+ *
+ * A rejected key rarely comes back as tidy JSON: a gateway in front of the
+ * provider answers with an HTML error page, and the SDK puts that entire page
+ * in `error.message`. Users were seeing raw markup in the toast.
+ */
+export function describeAiError(
+  error: unknown,
+  translate: (kind: AiErrorKind) => string = (kind) => en.aiErrors[kind]
+): string {
+  const kind = aiErrorKind(error)
+  if (kind) return translate(kind)
+  const text = rawMessage(error)
   return text.length > 200 ? `${text.slice(0, 200)}...` : text
 }

src/lib/ai.ts

@@ -2,7 +2,8 @@ import 'server-only'
 import { aiSetup } from '@/features/integrations/Lib/ai'
 import { localeNames, type Locale } from '@/i18n/config'
 import OpenAI from 'openai'
-import { describeAiError } from '@/lib/ai-error'
+import { isAccountLimitError } from '@/lib/ai-error'
+import { aiErrorDescriber } from '@/lib/ai-error-message'
 import {
   ANTHROPIC_BASE,
   ANTHROPIC_VERSION,
@@ -142,7 +143,7 @@ async function chatCompletion(
     return response.choices[0]?.message?.content ?? ''
   } catch (error) {
     console.error('[ai] chat completion failed:', error)
-    throw new Error(describeAiError(error))
+    throw new Error((await aiErrorDescriber())(error))
   }
 }
 
@@ -176,7 +177,7 @@ export async function visionCompletion(
     return response.choices[0]?.message?.content ?? ''
   } catch (error) {
     console.error('[ai] vision completion failed:', error)
-    throw new Error(describeAiError(error))
+    throw new Error((await aiErrorDescriber())(error))
   }
 }
 
@@ -288,6 +289,85 @@ severity is 1-5 based on how widely reported the issue is (5 = extremely common/
   return chatCompletion(organizationId, systemPrompt, userPrompt)
 }
 
+/**
+ * The answer budget of a connection test. The reply is never read, so this
+ * is the smallest cap no model turns down: OpenAI's Responses API refuses
+ * one under 16 outright, and a cap of 1 made its GPT-5 models fail other
+ * tools' health checks. `completionTuning` adds the thinking headroom on top
+ * for the models that need it, as it does for a real request. The cap is a
+ * ceiling; the test costs the few tokens of a one-word reply.
+ */
+const PROBE_TOKENS = 16
+
+/**
+ * OpenAI's GPT-5 family answers a cap that ran out before the reply was
+ * whole with a 400 where the other models return a cut-off reply. Either way
+ * the key was accepted, the account could pay and the model ran.
+ */
+function cutOffByTokenCap(error: unknown): boolean {
+  return (
+    error instanceof OpenAI.APIError &&
+    error.status === 400 &&
+    /max_tokens or model output limit was reached/i.test(error.message)
+  )
+}
+
+/** How long a connection test waits on the model before it gives up. */
+const PROBE_TIMEOUT_MS = 30_000
+
+/**
+ * A failure that says nothing about the key: the provider was busy, down or
+ * slow, or never answered. A real rate limit is one of these; an account
+ * that cannot pay is not, though it arrives under the same status.
+ */
+function providerTrouble(error: unknown): boolean {
+  if (!(error instanceof OpenAI.APIError)) return false
+  if (error.status === undefined || error.status >= 500) return true
+  return error.status === 429 && !isAccountLimitError(error)
+}
+
+/**
+ * One tiny completion on the model the workshop chose, for a connector's
+ * connection test. Listing models proves the key exists; only a completion
+ * proves the account has credit and the key may use that model, and without
+ * it a test passed on a connection where every request then failed. It goes
+ * out the way real requests do, and a failure is worded the way theirs is.
+ *
+ * Any reply passes, an empty or cut-off one included: what it says is not
+ * the question. No temperature is sent, because several models refuse one.
+ *
+ * It is asked once and not for long. The SDK would otherwise try a refusal
+ * three times and wait as long as the provider told it to, with somebody
+ * watching a spinner. And a failed test switches the connection off, so when
+ * the provider was merely busy the answer is `inconclusive`: the caller
+ * falls back to the check it had before rather than turn off an AI
+ * connection that works.
+ */
+export async function probeAiModel(
+  config: AiConfig
+): Promise<{ ok: boolean; message?: string; inconclusive?: boolean }> {
+  const client = createClient(config)
+  try {
+    await client.chat.completions.create(
+      {
+        model: config.model,
+        messages: [{ role: 'user', content: 'Reply with OK.' }],
+        ...completionTuning(config, PROBE_TOKENS),
+      },
+      { maxRetries: 0, timeout: PROBE_TIMEOUT_MS }
+    )
+    return { ok: true }
+  } catch (error) {
+    if (cutOffByTokenCap(error)) return { ok: true }
+    console.error('[ai] connection test failed:', error)
+    return {
+      ok: false,
+      message: (await aiErrorDescriber())(error),
+      ...(providerTrouble(error) && { inconclusive: true }),
+    }
+  }
+}
+
 export async function testAiConnection(organizationId: string): Promise<boolean> {
   const config = await getAiConfig(organizationId)
   const client = createClient(config)