“How would you rate your knowledge of techonolgy X out of 10?” Is a terrible question, and here’s why.
It’s entirely ambiguous, and depending on which interpretation you give, there’s really no wrong answer.
On an obsessively literal level there are reasonable interpretations of the question such that most candidates would say 10, not out any hubris, but because the 10th percentile isn’t really that hard to attain – 10% of people do it. So if the candidate interprets the question to mean that we have 10 equally sized buckets and we’re plotting anybody who’s ever used the tech once into buckets, it’s expected they’d be 8+. However, if it’s a normal distribution, and each bucket is 1 standard deviation, then 1 in 3million people is a 10. Pretty big distinction.
On a practical level the candidate knows it’s a poorly-defined question. A shrewd candidate is going to have to play the guessing game to determine what answer you want to hear. On a phone screen, a recruiter might reject you for saying you only have a 6 out of 10 in networking, whereas some interviewer might declare that you need over a dozen of years experience to get a 7/10.
So the question is really — can the candidate guess what I want to hear from them? And then the question is — is that what we’re selecting for in interviews?