Coding Challenges: A necessity or counterproductive?

Google, Facebook, Amazon, Microsoft and many other technology companies use them during their application process. Coding challenges, algorithmic programming tasks that try to reveal how good a software engineer is at finding efficient and scalable solutions.

During the technical interview process, an applicant is asked to solve a problem on a whiteboard on a computer, while the interviewer examines the work of the applicant. When the candidate performs well during the interview and coding challenges, a job offer can be expected soon after.

However, nowadays more and more questions are being raised whether or not these coding challenges are actually effective in finding the best developers for the job position. Candidates question the relevance of these challenges, experience high levels of stress and anxiety. Although companies believe that these technical interviews offer a “reasonably consistent evaluation of problem-solving ability, communication skills, and preparedness”, it is also believed that the current process fails in identifying good developers but the process is good in identifying who can perform well in an academic environment.

In this four part series I will try and shine a light on the practice of coding challenges in technical interviews. In this part I will investigate the effects that coding challenges have on developers going through a technical interview. In the next part I will look at the benefits companies experience with coding challenges and if it is actually effective. The third part will cover the design of these challenges and how to prepare for them, and finally in the fourth part of this series, I am going to look to the future of technical interviews and how they can be improved.

The history

It is not clear when and who started with coding challenges during technical interviews, however the consensus is that it started in the 90s when testing code was expensive and pen and paper, chalkboards and eventually whiteboards were used to test code. Tech companies like Microsoft started with brain teasers during job interviews to assess the skills of a potential employee. Interviewees were asked to solve simple logic puzzles and were subsequently scored on their solutions.

Gradually this evolved into proper coding challenges where applicants would be asked to solve more complex problems either on a whiteboard or on a computer. These problems would range from reversing a binary tree to perfectly shuffling a deck of cards. Entire books have been written on these exercises and courses have been created where aspiring developers can learn how to tackle these challenges.

The experience

How these interviews are set up or which types of questions that are being asked usually differ between tech companies. However, the experience from the interviewee’s point of view is usually not that different. The potential developer is put under a timed and supervised pressure to solve a difficult problem on a whiteboard or on a computer. Once the candidate is done, their solution is then scored based on their performance and the efficiency of their solution.

Unfortunately, these types of technical interviews can create a stressful experience for the candidate. The conditions in which the participant is placed closely resembles the Trier Social Test. The Trier Social Test is a test used by psychologists designed for the sole purpose of inducing stress in a candidate and seeing how they respond to the experienced stress.

In this test the participant is asked to present a prepared interview-style presentation about a given subject while performing mental arithmetic in front of a panel that does not offer any feedback. The aim of the test is to measure acute stress under laboratory conditions. When placing this in the context of a technical interview, it is clear to see that companies are not looking for which candidate is the best in solving a problem but who is the best in coping in a stressful environment.

A recent study performed at the NC State University found that, when a candidate is tasked with solving a problem with an interviewer present, almost two thirds of the participants were not able to complete the task in the given time. When they were asked to solve the problem in a private setting without the interviewer present, only one third of the participants failed in solving the challenges in the same amount of time. The same study also found that more than half of the participants were able to find a solution in less than ten minutes when asked to solve it without someone else present supervising their work.

Relevance

An often voiced concern about the technical interviews process by developers is that the presented coding challenges by interviewers are not relevant to the actual job or work. In a study that looked at the sentiment of developers towards the technical interview process, one developer mentioned that the solution they made for the problem in the given time-frame became more representative of their experience instead of their actual experience. The developer felt that their CV and what they had learned over the years became totally irrelevant when they had to do coding challenges at a job interview.

On the other hand, proponents of the technical interview said in the same study that understanding fundamental algorithms is, in their opinion, more favorable than understanding the myriad of modern frameworks. Although both sides of this discussion can be found in forum threads online, most of the time you will find the number of times stated that the current technical interview process is not relevant outweighs the number of times stated  that coding challenges were useful and relevant for their eventual work.

Conclusion

Coding challenges during the technical interview process have become a cornerstone of the solicitation process in the field of software engineering. Unfortunately, it has turned out to be a procedure to reliably induce stress in the candidates in a manner that very closely mimics the setup of the Trier Social Test. Companies are no longer testing the quality of the interviewee but they are testing how well the candidate can perform in an acute stress situation. Furthermore, the relevance of these coding challenges are questioned more and more. These observations are clearly indicating that for candidates of technical interviews, the current practices are lacking or faulty and should be changed.

This is part one of a four part series in which I try to investigate the world of coding challenges during technical interviews. In the next part I will look at the process from the point of view of the companies.

References

https://pubmed.ncbi.nlm.nih.gov/28229114/
https://doi.org/10.1109/VLHCC.2019.8818836
https://doi.org/10.1109/VLHCC.2019.8818836
https://dl.acm.org/doi/10.1145/3368089.3409712
https://betterprogramming.pub/a-history-of-coding-interviews-23b5e8f9c92f
https://www.goodreads.com/book/show/25707092-cracking-the-coding-interview
https://dev.to/lynnetye/engineering-whiteboard-interviews-yay-or-nay-3hko
https://doi.org/10.7717/peerj-cs.173