SIOP Conference Week 14th-17th April 2021 Panel Discussion

Personnel Selection in International Context: Current and Ongoing Challenges

 

I just posted on Linkedin about my session at the virtual SIOP 2021 conference.

Chair/Moderator Bharati B. Belwalkar, Ph.D., is based in New Orleans. She assembled this international panel, prepared the session, and choreographed our inputs. Please find below the session abstract and some session notes I prepared in advance.

 

Abstract

In this panel discussion, four selection and assessment practitioners will discuss their perspectives on and approaches to designing and implementing assessments internationally and the impact of COVID-19 pandemic on international testing.

Typically, developing and implementing assessments in international context requires careful considerations to various (culture, legal and technological constraints) factors. Unfortunately, organizations have very limited helpful resources for guidance when it comes to global selection and assessment practices. Additionally, testing practices across most international organizations are dealing with COVID crisis. We propose this panel discussion, among four experience practitioners, around designing and implementing assessments internationally and addressing the impact of COVID-19 on international testing.

 

How do you generally account for international differences and their implications on the content of the selection system?

• Legal challenges:
  • US is most litigious so that US validation study is critical if used there– less so in other jurisdictions.
  • Feedback mandatory in NL and Nordics – how much? Data Access requests for meaningful info; best practice is/was in-depth conversation; move towards simple and interactive candidate and assessor reports.
  • Automated feedback/debrief ‘forced feedback’ anecdote – dissuaded from use of Percentile score for cognitive test.
• Linguistic and cultural differences:
  • Develop items without idioms.
  • Make UKE and USE compatible.
  • Aim at item level equivalence from the outset; in an iterative process identify and replace or refine ‘howler’ items.
  • Use an entirely non-verbal cognitive test such as HUCAMA Reasoning Factor with Diagrammatic Matrices, Number Series and Spatial Reasoning components; use geographical norms to account for regional mean ability tests score differences.

What are your general recommendations for effective global selection and assessment?  

• Despite all of the hype generated by new AI-based predictive hiring and assessment tools, Handler (2021) data clearly indicates that the tried-and-true methods on which the industry was founded still dominate the market.When it comes to the most commonly offered assessment types, personality and cognitive assessments still dominate the market, being offered by 47% and 35% of all vendors respectively (the highest rates of any assessment type tracked).
• Investigate AI and new offerings but check hyperbole.
• Bright (GMA) * Motivated (GFP) = Success.
• Standardized criterion measures are key in addition to standardized predictors.
• interestingly a client study in the US replicated well outside UK so that cross-validated algorithms were used world-wide (however NA had highest validities closely followed by Europe; Asia moderate validity and low validity in Middle East / Africa).

Thanks again to Bharathi and the fellow panelists.

Please connect and contact me on Linkedin for further details!

Dr Rainer H. Kurz

C. Psychologist

London