Contacting HR over possible errors in test

[M&ALawyer]

I recently completed a verbal reasoning test for a well known City firm. This test had ambiguously worded questions, which led to multiple potential answers. I have researched the definitions used for words in the questions, and consulted others - and I am confident in my evaluation. I would like to reiterate this is over the choice of words in the question - rather than the material itself.

I have probably completed 50+ of these verbal reasoning/Watson Glaser tests, and not had any issues with the wording of the questions previously. To my knowledge, this test appeared to be designed in-house rather than a company such as Pearson.

I am considering writing to HR, explaining with clear examples how the wording of the questions does not lead to an answer. However, equally – I don't want to have my application viewed negatively for raising these issues!

Any advice is much appreciated! Thanks.

 

[Jaysen]

I recently completed a verbal reasoning test for a well known City firm. This test had ambiguously worded questions, which led to multiple potential answers. I have researched the definitions used for words in the questions, and consulted others - and I am confident in my evaluation. I would like to reiterate this is over the choice of words in the question - rather than the material itself.

I have probably completed 50+ of these verbal reasoning/Watson Glaser tests, and not had any issues with the wording of the questions previously. To my knowledge, this test appeared to be designed in-house rather than a company such as Pearson.

I am considering writing to HR, explaining with clear examples how the wording of the questions does not lead to an answer. However, equally – I don't want to have my application viewed negatively for raising these issues!

Any advice is much appreciated! Thanks.

Sounds like a nightmare test. Personally, I think it would be worth waiting until after you have learned about the outcome if you haven't already. I don't think it'll be a problem providing feedback on the test, particularly if you word your email in the right way, but I'd still play it safe.

 

[OB]

I recently completed a verbal reasoning test for a well known City firm. This test had ambiguously worded questions, which led to multiple potential answers. I have researched the definitions used for words in the questions, and consulted others - and I am confident in my evaluation. I would like to reiterate this is over the choice of words in the question - rather than the material itself.

I have probably completed 50+ of these verbal reasoning/Watson Glaser tests, and not had any issues with the wording of the questions previously. To my knowledge, this test appeared to be designed in-house rather than a company such as Pearson.

I am considering writing to HR, explaining with clear examples how the wording of the questions does not lead to an answer. However, equally – I don't want to have my application viewed negatively for raising these issues!

Any advice is much appreciated! Thanks.

I'm not sure confronting the firm that their test in incorrect would be the best course of action, and comes off a bit arrogant in their eyes? I'm sure they wouldn't send out any test which was incorrect and its designed to test how you interpret the questions and assumptions/conclusions. Maybe feedback that it was more ambiguous than other tests you've taken if theres a good opportunity to provide feedback, and if this is the first cycle they've introduced the test. But you'd have to be very careful with how you worded any email if you want to show examples of how you think their test should've been phrased...

 

[M&ALawyer]

I'm not sure confronting the firm that their test in incorrect would be the best course of action, and comes off a bit arrogant in their eyes? I'm sure they wouldn't send out any test which was incorrect and its designed to test how you interpret the questions and assumptions/conclusions. Maybe feedback that it was more ambiguous than other tests you've taken if theres a good opportunity to provide feedback, and if this is the first cycle they've introduced the test. But you'd have to be very careful with how you worded any email if you want to show examples of how you think their test should've been phrased...

Sounds like a nightmare test. Personally, I think it would be worth waiting until after you have learned about the outcome if you haven't already. I don't think it'll be a problem providing feedback on the test, particularly if you word your email in the right way, but I'd still play it safe.

Thank you for both your replies.

In terms of what I hope to achieve – I passed the test and am in the next stage of the process – so I am not really affected by this. My main intention is to not make the process more stressful than it has to be for other current (and future) candidates, as I panicked a bit when I was confronted with questions with no one answer. In my view, it's unfortunate that suggestions to improve a recruitment process for others has the potential to come off as arrogant/negative – but as you both correctly highlight the best course of action may be to just ignore it for now, especially as there is not really a feedback section.

Apologies for any vagueness/confusion – it is challenging to explain precisely what I am talking about because if I mention content of the questions, or the firm, it will have implications on the confidentiality of the test. However, to expand a bit, certain economic terms and processes were used in the questions which can have multiple (completely valid) definitions, and result in multiple answers. I am aware of this as I have a work background in finance/economics.

 

[Jaysen]

Thank you for both your replies.

In terms of what I hope to achieve – I passed the test and am in the next stage of the process – so I am not really affected by this. My main intention is to not make the process more stressful than it has to be for other current (and future) candidates, as I panicked a bit when I was confronted with questions with no one answer. In my view, it's unfortunate that suggestions to improve a recruitment process for others has the potential to come off as arrogant/negative – but as you both correctly highlight the best course of action may be to just ignore it for now, especially as there is not really a feedback section.

Apologies for any vagueness/confusion – it is challenging to explain precisely what I am talking about because if I mention content of the questions, or the firm, it will have implications on the confidentiality of the test. However, to expand a bit, certain economic terms and processes were used in the questions which can have multiple (completely valid) definitions, and result in multiple answers. I am aware of this as I have a work background in finance/economics.

As you've passed the test, I don't really see an issue with you providing feedback to the firm. The way you've explained this here sounds like you are just hoping to clarify things for future candidates.

If you do decide to go ahead, and need someone to check over the way you phrase an email, feel free to drop me a PM. (You can anonymise the firm etc. if you'd like to.)

 

[M&ALawyer]

As you've passed the test, I don't really see an issue with you providing feedback to the firm. The way you've explained this here sounds like you are just hoping to clarify things for future candidates.

If you do decide to go ahead, and need someone to check over the way you phrase an email, feel free to drop me a PM. (You can anonymise the firm etc. if you'd like to.)

Thanks Jaysen – I will drop you a PM if I go ahead with sending an email, and find the time to do so.

All the best!

 

[Jessica Booker]

Just because it isn’t designed by Pearson or a big provider, doesn’t mean it hasn’t been designed professionally by a qualified BPS professional.

my take on this is that it’s likely 100s if not 1000s of people have taken the test before you. It will have also been tested out within the firm itself too. If there was any aspect of there being a “lack of clear outcome” from particular questions, it would have been picked up in the design process. I suspect that they are either in there correctly as part of the assessment or that it isn’t necessarily how you perceive it. The there isn’t a clear answer it is probably the “cannot determine” type answer usually provided in verbal reasoning test options.

You can highlight your concerns to HR but be prepared for them to tell you that you are wrong. That could easily be the reality here.

 

[M&ALawyer]

Just because it isn’t designed by Pearson or a big provider, doesn’t mean it hasn’t been designed professionally by a qualified BPS professional.

my take on this is that it’s likely 100s if not 1000s of people have taken the test before you. It will have also been tested out within the firm itself too. If there was any aspect of there being a “lack of clear outcome” from particular questions, it would have been picked up in the design process. I suspect that they are either in there correctly as part of the assessment or that it isn’t necessarily how you perceive it. The there isn’t a clear answer it is probably the “cannot determine” type answer usually provided in verbal reasoning test options.

You can highlight your concerns to HR but be prepared for them to tell you that you are wrong. That could easily be the reality here.

Thank you Jessica for your response.

For clarity, I have adapted one question I have issue with in the test. I welcome any thoughts with my interpretation!

Statement in the passage: "....the market is more competitive"
Question: Has there been an increase of similar products in the market. Yes, no, cannot say.

Answer(s):
Yes - competition can be caused by more products in the market
cannot say - the market could be more competitive because of information availability and/or other factors (e.g reduction in tariffs/subsidies).

For reference please see: https://www.investopedia.com/ask/answers/042115/what-factors-influence-competition-microeconomics.asp#:~:text=The number of sellers also,market is considered a monopoly.

I selected yes as I know this is what they are looking for (and my test score implies that I likely was scored correctly) - but this is not actually correct.

In any case, I don't believe the question should be included - this isn't verbal reasoning but application of economics?

 

[Jessica Booker]

Thank you Jessica for your response.

For clarity, I have adapted one question I have issue with in the test. I welcome any thoughts with my interpretation!

Statement in the passage: "....the market is more competitive"
Question: Has there been an increase of similar products in the market. Yes, no, cannot say.

Answer(s):
Yes - competition can be caused by more products in the market
cannot say - the market could be more competitive because of information availability and/or other factors (e.g reduction in tariffs/subsidies).

For reference please see: https://www.investopedia.com/ask/answers/042115/what-factors-influence-competition-microeconomics.asp#:~:text=The number of sellers also,market is considered a monopoly.

I selected yes as I know this is what they are looking for (and my test score implies that I likely was scored correctly) - but this is not actually correct.

In any case, I don't believe the question should be included - this isn't verbal reasoning but application of economics?

I really can’t comment on what is right or wrong based on this format.

You might be right, there could be an error and therefore feel free to flag this to HR. However, knowing how these things are designed and monitored, I’d bet money on it more being about your interpretation of this rather than something being fundamentally wrong.

For instance, you already saying “I know this is what they are looking for” is concerning to me. They aren’t “looking for something”. You are also making some big assumptions that you were correct on this answer just because you passed the test too.

 

[OB]

Thank you Jessica for your response.

For clarity, I have adapted one question I have issue with in the test. I welcome any thoughts with my interpretation!

Statement in the passage: "....the market is more competitive"
Question: Has there been an increase of similar products in the market. Yes, no, cannot say.

Answer(s):
Yes - competition can be caused by more products in the market
cannot say - the market could be more competitive because of information availability and/or other factors (e.g reduction in tariffs/subsidies).

For reference please see: https://www.investopedia.com/ask/answers/042115/what-factors-influence-competition-microeconomics.asp#:~:text=The number of sellers also,market is considered a monopoly.

I selected yes as I know this is what they are looking for (and my test score implies that I likely was scored correctly) - but this is not actually correct.

In any case, I don't believe the question should be included - this isn't verbal reasoning but application of economics. Perhaps this is intentional (and they're expecting the participant to put 'cannot say') - but it would seem a very harsh given most law applicants probably haven't studied economics!

Perhaps I am overthinking it and most applicants won't consider this – but it did get to me slightly when I was taking the test, and I am sure others must have felt the same.

Some questions tell you not to use any outside knowledge and purely rely on the words in the statement. If no mention to similar products is made in the initial statement, then the answer would be 'cannot say'?

 

[M&ALawyer]

I really can’t comment on what is right or wrong based on this format.

You might be right, there could be an error and therefore feel free to flag this to HR. However, knowing how these things are designed and monitored, I’d bet money on it more being about your interpretation of this rather than something being fundamentally wrong.

For instance, you already saying “I know this is what they are looking for” is concerning to me. They aren’t “looking for something”. You are also making some big assumptions that you were correct on this answer just because you passed the test too.

Fair points - I think ultimately my main concern ultimately comes down to loose use of technical terms – which I believe is a bit beyond the jurisdiction of VR tests. Perhaps, as you highlight it is fair game and designed intentionally - but this would be quite unusual in my experience.

My use of "I know what they're looking for" implies my guess is they are expecting an answer with a low level definition of competition (more products).

 

[M&ALawyer]

Some questions tell you not to use any outside knowledge and purely rely on the words in the statement. If no mention to similar products is made in the initial statement, then the answer would be 'cannot say'?

I think this is true up to a point.

Defining competition is not "outside knowledge". If the market which the product is being sold becomes more competitive, then this implies that there are more similar products in the market; but this is not absolute.

 

[Legal_rawn]

Thank you Jessica for your response.

For clarity, I have adapted one question I have issue with in the test. I welcome any thoughts with my interpretation!

Statement in the passage: "....the market is more competitive"
Question: Has there been an increase of similar products in the market. Yes, no, cannot say.

Answer(s):
Yes - competition can be caused by more products in the market
cannot say - the market could be more competitive because of information availability and/or other factors (e.g reduction in tariffs/subsidies).

For reference please see: https://www.investopedia.com/ask/answers/042115/what-factors-influence-competition-microeconomics.asp#:~:text=The number of sellers also,market is considered a monopoly.

I selected yes as I know this is what they are looking for (and my test score implies that I likely was scored correctly) - but this is not actually correct.

In any case, I don't believe the question should be included - this isn't verbal reasoning but application of economics?

Hi! from an economics degree perspective, reduction in tariffs or subsidies increase competition because they encourage entry into the market. Market competition is based on the concentration of firms/products in the market. Note that it says products not firms as well so I do this the answer is yes. Again I could be wrong but from someone who takes an econ degree and has done lots of modules on this I would say yes seems to be the right answer. I don't know if this is any help and again I could be wrong!

 

[Jessica Booker]

Some questions tell you not to use any outside knowledge and purely rely on the words in the statement. If no mention to similar products is made in the initial statement, then the answer would be 'cannot say'?

completely this! You have to only go by the information you are presented with in most of these questions. This is why I always go wrong with psycho metrics, because I am always trying to add in factors that I think could happen in scenarios.

 

[Jessica Booker]

Fair points - I think ultimately my main concern ultimately comes down to loose use of technical terms – which I believe is a bit beyond the jurisdiction of VR tests. Perhaps, as you highlight it is fair game and designed intentionally - but this would be quite unusual in my experience.

My use of "I know what they're looking for" implies my guess is they are expecting an answer with a low level definition of competition (more products).

that’s still a dangerous guess in my opinion

 

[M&ALawyer]

Hi! from an economics degree perspective, reduction in tariffs or subsidies increase competition because they encourage entry into the market. Market competition is based on the concentration of firms/products in the market. Note that it says products not firms as well so I do this the answer is yes. Again I could be wrong but from someone who takes an econ degree and has done lots of modules on this I would say yes seems to be the right answer. I don't know if this is any help and again I could be wrong!

Thanks for your input Rhiannon - its good to have an economists perspective!

Reductions in tariffs and increased subsidies also can make goods+services more competitively priced in a market; as competition is not solely based on concentration of products but also price (if I am not mistaken). I thought I was overthinking this for the purpose of a VR test; which is why I opted for yes for the reasons you mentioned.

completely this! You have to only go by the information you are presented with in most of these questions. This is why I always go wrong with psycho metrics, because I am always trying to add in factors that I think could happen in scenarios.

I think it might be a lesson not to dwell too much on technicalities for these tests. But I think selection of words should still be factored in with greater care. My points might sound incredibly minor/pernickety in the scheme of things – but say a candidate missed the benchmark by one correct answer because they had the same concerns I had (which in my opinion are reasonable); I am not sure it is just?

 

[Legal_rawn]

Thanks for your input Rhiannon - its good to have an economists perspective!

Reductions in tariffs and increased subsidies also can make goods+services more competitively priced in a market; as competition is not solely based on concentration of products but also price (if I am not mistaken). I thought I was overthinking this for the purpose of a VR test; which is why I opted for yes for the reasons you mentioned.



I think it might be a lesson not to dwell too much on technicalities for these tests. But I think selection of words should still be factored in with greater care. My points might sound incredibly minor/pernickety in the scheme of things – but say a candidate missed the benchmark by one correct answer because they had the same concerns I had (which in my opinion are reasonable); I am not sure it is just?

I think you did the completely right thing. These sort of questions aren't expecting you to have debated the definition of competition and how it should be measured. I do understand how it can be frustrating when you are doing them though

 

[M&ALawyer]

I think you did the completely right thing. These sort of questions aren't expecting you to have debated the definition of competition and how it should be measured. I do understand how it can be frustrating when you are doing them though

Exactly – I think the tests are stressful enough without having to internally debate with yourself what selected definition you should be used for a word like competition. Especially as its timed!

All the best!

 

[Jessica Booker]

Thanks for your input Rhiannon - its good to have an economists perspective!

Reductions in tariffs and increased subsidies also can make goods+services more competitively priced in a market; as competition is not solely based on concentration of products but also price (if I am not mistaken). I thought I was overthinking this for the purpose of a VR test; which is why I opted for yes for the reasons you mentioned.



I think it might be a lesson not to dwell too much on technicalities for these tests. But I think selection of words should still be factored in with greater care. My points might sound incredibly minor/pernickety in the scheme of things – but say a candidate missed the benchmark by one correct answer because they had the same concerns I had (which in my opinion are reasonable); I am not sure it is just?

I am still not convinced there is a technical error in language here. As stressed you are basing it your off comprehension of the definition of competition - not based on the information that is presented to you in the answer. To me this sounds like it is purposely ambiguous and that therefore a “cannot determine” is a good chance of being the right answer.

 

[Jessica Booker]

Exactly – I think the tests are stressful enough without having to internally debate with yourself what selected definition you should be used for a word like competition. Especially as its timed!

All the best!

you shouldn’t have to be having an internal debate. Typically the only debate will be based on the information you are presented with - try to avoid the external references/influences