Senior Technical Lead at HCLTech · HyrikoBack to jobsLocation:HyderabadType:Full-timePosted:Yesterday In this role, you will:
- Build and operate AI evaluation workflows that measure the quality of LLM outputs across chat,
summarization, recommendations, and agent-based features.
- Implement LLM-as-a-Judge and rubric-based evals to score outputs for correctness, relevance, grounding,
and consistency.
- Instrument LLM and agent workflows to capture traces, prompts and responses, metadata, and user
feedback.
- Support release readiness by running evals before launches and highlighting regressions or quality risks.
- Help define agent-specific evals (task completion, tool correctness, error recovery).
- Partner with AI engineers and AI platform teams to translate product requirements into evals criteria.
- Review eval results and recommend improvements.
- Contribute to system design for observability, retries, and logging.
Minimum Qualifications
- 4+ years of experience in data and AI-related fields such AI engineering, software development, ML
engineering, data science, or QA roles.
- We’re looking for someone with an eagerness and ability to learn new skills and solve dynamic problems
in an encouraging and expansive environment.
- Working across global teams to ensure alignment of product development.
- Strong Python skills.
- Applied knowledge of GenAI and RAG strategies, micro-services, recommendation systems, and context
engineering.
Don't want to miss the next one?
Subscribe to daily email alerts for roles matching your interests.
Familiarity of AI evaluation techniques, such as Golden datasets, LLM-as-judge, or rubric-based scoring.Experience with different LLM ecosystems (OpenAI, Anthropic, Gemini, etc.), RAG pipelines, vectordatabases (e.g., Pinecone, FAISS, Milvus, PostgreSQL).
- Proficiency in SQL and experience with at least one major data analytics platform, such as Hadoop,
- Experience with CI/CD or release validation workflows.
- Familiarity with telemetry and evaluation frameworks for AI agents.
- Experience working with data science teams on insights generation leveraging LLMs.
- Knowledge of project management, and productivity tools such as Wrike and Miro.
- Strong time management skills with the ability to collaborate across multiple teams.
- Able to balance competing priorities, long-term projects, and ad hoc requirements.
- Ability to work in a fast-paced, dynamic, constantly evolving business environment.
Preferred Qualifications
- Hands-on experience with Langfuse or similar tools for LLMs observability.
- Sound communication skills - expert at messaging domain and technical content, at a level appropriate
for the audience. Strong ability to gain trust with stakeholders and senior leadership.
- Familiarity with embeddings, retrieval algorithms, agents, and data modeling for vector development
- Other complementary technologies for distributed systems architecture and asynchronous messaging,
agent communication, and catching like RabbitMQ, Redis, and Valkey are preferred. Skill Requirements
Minimum Qualifications
- 4+ years of experience in data and AI-related fields such AI engineering, software development, ML
engineering, data science, or QA roles.
- We’re looking for someone with an eagerness and ability to learn new skills and solve dynamic problems
in an encouraging and expansive environment.
- Working across global teams to ensure alignment of product development.
- Strong Python skills.
- Applied knowledge of GenAI and RAG strategies, micro-services, recommendation systems, and context
- Familiarity of AI evaluation techniques, such as Golden datasets, LLM-as-judge, or rubric-based scoring.
- Experience with different LLM ecosystems (OpenAI, Anthropic, Gemini, etc.), RAG pipelines, vector
databases (e.g., Pinecone, FAISS, Milvus, PostgreSQL).
- Proficiency in SQL and experience with at least one major data analytics platform, such as Hadoop,
- Experience with CI/CD or release validation workflows.
- Familiarity with telemetry and evaluation frameworks for AI agents.
- Experience working with data science teams on insights generation leveraging LLMs.
- Knowledge of project management, and productivity tools such as Wrike and Miro.
- Strong time management skills with the ability to collaborate across multiple teams.
- Able to balance competing priorities, long-term projects, and ad hoc requirements.
- Ability to work in a fast-paced, dynamic, constantly evolving business environment.
Preferred Qualifications
- Hands-on experience with Langfuse or similar tools for LLMs observability.
- Sound communication skills - expert at messaging domain and technical content, at a level appropriate
for the audience. Strong ability to gain trust with stakeholders and senior leadership.
- Familiarity with embeddings, retrieval algorithms, agents, and data modeling for vector development
- Other complementary technologies for distributed systems architecture and asynchronous messaging,
agent communication, and catching like RabbitMQ, Redis, and Valkey are preferred.
Similar roles you might like
More openings like this one — take a look before you go.
Browse all jobs
