Jack Taylor retweeted
Any code that has no tests is broken code.
There are a few skills more important than learning how to evaluate LLM-powered applications.
Building an LLM demo is straightforward, but every professional developer can tell you how hard it is to build production-grade systems that work.
Here is how you can unit test your LLM-powered application (this works for agents, RAG, chatbots, anything):
1. pip install deepeval
2. write or generate a few LLM test cases
3. select the relevant metrics for testing
DeepEval is open source: github.com/confident-ai/deep…
You can use it to benchmark your application's performance against any criteria relevant to your use case. The library supports more than 14 research-backed metrics out of the box.
Look at the attached code: it tests the "relevancy" of an LLM chatbot's responses.
Here is something pretty cool:
You can use the library to generate synthetic test data based on your knowledge base to avoid writing test cases manually yourself. This is huge and saves a ton of time!
There's much more about DeepEval that I'll let you discover, but I also want to mention that it integrates natively with Pytest, so you can use it in CI/CD pipelines.
(In case you don't know, Pytest is *the* most popular unit-testing library in the Python ecosystem.)
Hi fellow testers
Currently revamping my CV and was wondering if any other lead/senior testers would be willing to share theirs with me?
Would be really helpful as I modify mine to be more up to date with the world of modern testing!
20
Any testers out there in a senior role who would be able to take a look at my CV and make some suggestions?
Ive been applying for lots of stuff and not heard back from anything. Admiteddly I am aiming high (happy in current role) but worried im making some glaring errors...
4
305
Seeing as how everyone is at it, what are the coolest AI trends/tools in Testing? Anything I can wow my team with? 😮
103
Jack Taylor retweeted
Just found this model on the internet, I'm going to use it in a talk and claim it's mine.
Felllow testers, an opportunity in my team has arisen! The job spec focuses heavily on automation and whilst that's of course necessary, I would love some applicants that have a genuine passion for our profession!
https://teapawt.net/t.co/NUap373jSr
#testing #softwaretesting #automation
64
Anyone got some really good examples of exploratory test sessions or some ideas to freshen them up? Could really use some inspiration!
#exploratorytesting
Just had a meeting with the UAT team on my latest project only to discover they had built/are building a selenium regression suite. They wanted to know if we had any scripts we could leverage...
I have never seen a UAT team with their own automation suite - is it just me?!
#uat
Thought it was time to breath some life into my CV and there is definitely a big hole in my mobile testing experience.
Does anyone have any recommendations of courses/reading material that would help close this? Would be really cool to start a side project.
#testautomation
Best new software testing apps/tools I should be looking at towards year end?
Really want to find something funky, but so far I feel like im forcing the issue...
#testing #agile #softwaretesting
1
Hey QAs, whats the most interesting new area of our field for you atm? I want to prepare a talk for our testers network and im looking for inspiration!
So the interviews for a senior test role I did this week went well, but a familiar story...
Great technical UI testing skills, but no non-functional or exploratory experience and no real community engagement. I feel like so many people dont explore this great profession enough!
Jack Taylor retweeted
#Tools are helpful to testers in performing checks and not built with an intent to replace them.
Reflecting on the evolution of #testing tools and what lies ahead...
#softwaretesting #automationtesting #automationtools #automation
So have a couple of interviews to do tomorrow for a Senior test role. Its been a while!
Any great questions or resources you all would recommend?
1
What are the most interesting/innovative things youve done or seen in an exploratory test session?
Ive been running my sessions for some time now and im trying to inspire a bit of creativity into the team!
Love this card from @RichardDarani and love the extra touches !
Thouroughly recommend richarddarani.co.uk/ if youre in need of a card still!
Anyone have any interesting stories of #exploratorytesting in large groups?
I feel the more chaos the better! For example, in my last meeting I thought we were all working off the same test env. but we hadnt. As a result, we unexpectedly found some inconsistencies in test data
Jack Taylor retweeted
Can you help @jack_taylor24 out over on The Club? 🙏🏻
Should statefulness be part of the test itself rather than the test data? 🤔
buff.ly/3tsSUqJ









