I'm open to the possibility that the traditional process is still the best way to select a colleague. But has anyone studied this question? Is anyone familiar with the personnel selection literature as applied to high-end positions like scientists, professors and researchers?
Here's a highly cited article that a quick search turned up. GMA = "General Mental Ability" :-)
The Validity and Utility of Selection Methods in Personnel Psychology: Practical and Theoretical Implications of 85 Years of Research Findings
This article summarizes the practical and theoretical implications of 85 years of research in personnel selection. On the basis of meta-analytic findings, this article presents the validity of 19 selection procedures for predicting job performance and training performance and the validity of paired combinations of general mental ability (GMA) and Ihe 18 other selection procedures. Overall, the 3 combinations with the highest multivariate validity and utility for job performance were GMA plus a work sample test (mean validity of .63), GMA plus an integrity test (mean validity of .65), and GMA plus a structured interview (mean validity of .63). A further advantage of the latter 2 combinations is that they can be used for both entry level selection and selection of experienced employees. The practical utility implications of these summary findings are substantial. The implications of these research findings for the development of theories of job performance are discussed.
... As noted above, GMA is also an excellent predictor of job-related learning. It has been found to have high and essentially equal predictive validity for performance (amount learned) in job training programs for jobs at all job levels studied. In the U.S. Department of Labor research, the average predictive validity performance in job training programs was .56 (Hunter & Hunter, 1984, Table 2); this is the figure entered in Table 2. Thus, when an employer uses GMA to select employees who will have a high level of performance on the job, that employer is also selecting those who will learn the most from job training programs and will acquire job knowledge faster from experience ... Because of its special status, GMA can be considered the primary personnel measure for hiring decisions, and one can consider the remaining 18 personnel measures as supplements to GMA measures.