Jack Taylor (@jack_taylor24) header
Jack Taylor (@jack_taylor24) profile picture

Senior Tester working in Fin Tech, guitarist in thrash band Bangover and wearer of sentient beard.

Brighton/London, UK
Joined April 2012
Photo by Jack Taylor (@jack_taylor24)Photo by Jack Taylor (@jack_taylor24)Photo by Jack Taylor (@jack_taylor24)Photo by Jack Taylor (@jack_taylor24)Photo by Jack Taylor (@jack_taylor24)Photo by Jack Taylor (@jack_taylor24)Photo by Jack Taylor (@jack_taylor24)Photo by Jack Taylor (@jack_taylor24)Photo by Jack Taylor (@jack_taylor24)
Jack Taylor retweeted
Any code that has no tests is broken code. There are a few skills more important than learning how to evaluate LLM-powered applications. Building an LLM demo is straightforward, but every professional developer can tell you how hard it is to build production-grade systems that work. Here is how you can unit test your LLM-powered application (this works for agents, RAG, chatbots, anything): 1. pip install deepeval 2. write or generate a few LLM test cases 3. select the relevant metrics for testing DeepEval is open source: github.com/confident-ai/deep… You can use it to benchmark your application's performance against any criteria relevant to your use case. The library supports more than 14 research-backed metrics out of the box. Look at the attached code: it tests the "relevancy" of an LLM chatbot's responses. Here is something pretty cool: You can use the library to generate synthetic test data based on your knowledge base to avoid writing test cases manually yourself. This is huge and saves a ton of time! There's much more about DeepEval that I'll let you discover, but I also want to mention that it integrates natively with Pytest, so you can use it in CI/CD pipelines. (In case you don't know, Pytest is *the* most popular unit-testing library in the Python ecosystem.)
13
50
7
277
33,406
Hi fellow testers Currently revamping my CV and was wondering if any other lead/senior testers would be willing to share theirs with me? Would be really helpful as I modify mine to be more up to date with the world of modern testing!
20
Any testers out there in a senior role who would be able to take a look at my CV and make some suggestions? Ive been applying for lots of stuff and not heard back from anything. Admiteddly I am aiming high (happy in current role) but worried im making some glaring errors...
4
305
Seeing as how everyone is at it, what are the coolest AI trends/tools in Testing? Anything I can wow my team with? 😮
103
Just found this model on the internet, I'm going to use it in a talk and claim it's mine.
15
14
1
96
12,500
Felllow testers, an opportunity in my team has arisen! The job spec focuses heavily on automation and whilst that's of course necessary, I would love some applicants that have a genuine passion for our profession! https://teapawt.net/t.co/NUap373jSr #testing #softwaretesting #automation
64
Anyone got some really good examples of exploratory test sessions or some ideas to freshen them up? Could really use some inspiration! #exploratorytesting
Just had a meeting with the UAT team on my latest project only to discover they had built/are building a selenium regression suite. They wanted to know if we had any scripts we could leverage... I have never seen a UAT team with their own automation suite - is it just me?! #uat
Well thats something I never thought I would see. The Student Loans Company owing ME money!
Thought it was time to breath some life into my CV and there is definitely a big hole in my mobile testing experience. Does anyone have any recommendations of courses/reading material that would help close this? Would be really cool to start a side project. #testautomation
Best new software testing apps/tools I should be looking at towards year end? Really want to find something funky, but so far I feel like im forcing the issue... #testing #agile #softwaretesting
1
Hey QAs, whats the most interesting new area of our field for you atm? I want to prepare a talk for our testers network and im looking for inspiration!
So the interviews for a senior test role I did this week went well, but a familiar story... Great technical UI testing skills, but no non-functional or exploratory experience and no real community engagement. I feel like so many people dont explore this great profession enough!
3
2
16
#Tools are helpful to testers in performing checks and not built with an intent to replace them. Reflecting on the evolution of #testing tools and what lies ahead... #softwaretesting #automationtesting #automationtools #automation
1
2
13
So have a couple of interviews to do tomorrow for a Senior test role. Its been a while! Any great questions or resources you all would recommend?
1
What are the most interesting/innovative things youve done or seen in an exploratory test session? Ive been running my sessions for some time now and im trying to inspire a bit of creativity into the team!
Love this card from @RichardDarani and love the extra touches ! Thouroughly recommend richarddarani.co.uk/ if youre in need of a card still!
1
1
1
Anyone have any interesting stories of #exploratorytesting in large groups? I feel the more chaos the better! For example, in my last meeting I thought we were all working off the same test env. but we hadnt. As a result, we unexpectedly found some inconsistencies in test data
1
1
1
Can you help @jack_taylor24 out over on The Club? 🙏🏻 Should statefulness be part of the test itself rather than the test data? 🤔 buff.ly/3tsSUqJ
2
2
Boss has posed this question: Service virtualization for testing: do they need statefulness? Pros/Cons? Or should this be a part of the tests themselves? My feeling is that statefulness should be part of the test itself rather then the test data. Any other arguments?
1