Test Engineer - AI Voice Systems

Deine Rolle im Team

Unsere Erwartungen an dich

Qualifikationen

• Practical command of LLM quality metrics - faithfulness, hallucination rate, relevance, plus at least one of BLEU / WER / CER - and the judgement to know what each does and does not prove.
• Prompt regression testing in practice: detecting quality drift across prompt edits and model version changes.
• Solid conventional test engineering: pytest, code coverage, test case design across positive, negative, boundary and regression cases; testcontainers or equivalent integration test tooling.
• CI/CD integration of automated test suites (Azure DevOps Pipelines, GitHub Actions) and comfort with Docker; working knowledge of REST APIs and SQL/PostgreSQL for test data and results.
• English C1, and willingness to work onsite in the Ingolstadt/Munich area, including hands-on test sessions in vehicles and on the test rack.

Erfahrung

• 5+ years in test automation / quality engineering with strong Python, including ownership of a test framework rather than only writing cases in someone else's.
• Hands-on experience evaluating LLM or other non-deterministic systems: building evaluation pipelines, LLM-as-judge patterns, embedding/cosine-similarity scoring, and reasoning about scoring reliability and inter-rater agreement.
• Experience with LangChain / LangGraph or equivalent, and with an LLM observability tool (LangFuse, Phoenix, Langsmith or comparable) for tracing, prompt versioning and evaluation runs.
• Experience integrating ASR and TTS services (Azure Speech, Whisper, Cerence, Google TTS or equivalent), including streaming/real-time behaviour.