Reader,

In Part 1, we talked about a bad tech assessment.

Let's see how it could be better, 1 skill at a time.

Recall that this was the list of skills they needed to assess for the role:

QA Automation (Playwright & POM)

These fall under the QA Automation umbrella in the 1st item.

Like I said in Part 1, they did a good job of testing the candidate on Playwright best practices.

But they can do better.

Instead of asking the candidate to putz around with AI to translate a Page Object Model file to another language, you know what would make more sense?

You can even have them add a new test, with a new element on the same page, so they can show that they know how to update a POM file.

The hiring team had a bogus test file that didn't actually run, though, because they only wanted the candidate to refactor it.

So this would require them to:

1. Find a page they can use for Playwright tests
2. Decide on a quick scenario
3. Choose locators for the elements they need for the test
4. Make a POM file to store those locators
5. Make sure the test passes consistently

Then they could give it to a candidate and have a LOT more options to play with for assessments.

Bogus tests are not useful to anyone.

Coding (TypeScript)

If the Playwright assessment asked the candidate to write a new test like I suggested above, this would be covered by default.

Moving on...

CI / CD

Crazy concept here, but...they could just add a GitHub Actions workflow file to the assessment.

Just a simple YAML file.

The one from the docs would suffice here:

Off the top of my head, they could:

• Break it, ask the candidate to fix it
• Ask the candidate what they would add or subtract from it, and why (good answers might be "add debugging information like Playwright traces" or "remove the HTML report in favor of a different reporting approach")
• Ask the candidate about corresponding changes they'd need to make in the Playwright config for stuff they wanna do in GitHub Actions -- this tells you they understand the tool well

The Rest of It -- Strategic Testing, Soft Skills, Mobile/Flutter/Appium

Strategic Testing (test plans, exploratory testing)

I often see this done as a conversational interview with a prewritten scenario.

Example: You have to test

They didn't have me run through any exercises like this, so I assume they didn't know how to test for it, or they didn't care about it as much as the automation skill set.

Soft Skills

Communication is something they can test throughout the interview process by watching how you interact with everyone.

Cross-functional influence and autonomy are hard to get a read on without stories about a candidate's past achievements.

These weren't part of the technical, but if they had wanted to work them into it, they could have had me pair with one of the interviewers to see how I handle it.

Flutter/Appium

Since Flutter isn't really something they consider a required skill, that can be something talked about in a conversational interview.

That way, they can get a gauge on what the candidate knows without having to test their ability to execute.

Besides, mobile test automation skills are hard to test in an interview setting. I haven't seen any take-home or live coding interviews that deal with mobile automation, since the technical setup creates a lot of friction.

In theory, you could show some Flutter code and ask someone to write a "fake" Appium script to test it, or modify an Appium script, but you likely won't end up running any of those tests.

So, what did we do here?

• Showed real examples of ways to test the skills on the req
• Explained why this hiring team's tests were insufficient for the req

Recruiter Takeaway:

It’s hard to know where a technical assessment falls short if you don’t understand the technology behind it or how the pieces fit together.

Start by partnering with your hiring manager to really understand the assessment and what it’s actually measuring. If your manager runs live coding sessions, make sure you’re aligned on how to introduce the exercise, what success looks like, and what the goal truly is.

Is the goal to get the code to run? Or is it to evaluate how a candidate thinks, communicates, and approaches problems?

Get agreement not just on the test itself, but on the purpose of the test and structure the interview to reflect what you’re trying to assess.

We get it. You weren’t hired for deep technical expertise. But technical fluency is something every recruiter can and should build. A few ways to do that:

• Talk with engineers in your network and ask for feedback
• Partner with engineers on your team to review and improve the assessment
• Combine that insight with your own research
• Build AI tools or use learning platforms like W3Schools to quickly build foundational knowledge

When you understand, shape, and control the technical assessment process, you’ll deliver better matches and a stronger experience for both clients and candidates. And you'll be in a position to advise your candidate on where to spend their time prepping.

Let's be real, nobody wants to prepare for an interview that doesn't actually happen.

If you’re part of our community and struggling on this piece...reach out.

Jaclyn is happy to help you 1:1, for free.

-Steven