Home
Blog
Quality Over Quantity

An Elegant Sufficiency: Quality Over Quantity in Diversity and Inclusion Survey Questions

19^th June 2024 by Mark Holt

We've run a lot of Diversity & Inclusion Surveys! One of the refrains that we constantly hear is that organisations want to "ask a very large number of questions in order to decide what is important".

When we fail to talk them down, these organisations tend to end up with a massive data set that identifies hundreds of minor issues (each of which needs to be triaged against it's neighbours). This always leads to an inability to see the wood for the trees with a corresponding impact on their ability to make make actionable change.

Instead, the success of an EDI survey lies not in the quantity of questions asked, but in the thoughtful selection and design of a small set of meaningful inquiries. Our questions benefit from years of evolution, so we'd strongly recommend starting with our default set of questions, adding one or two that target issues that the survey administrators may suspect, and building some experience.

If you're still not convinced, this article discusses why quality should be prioritized over quantity when it comes to D&I surveys, exploring the benefits of a concise and considered approach.

Keeping It Simple: The Benefits of Fewer Questions

Asking fewer, carefully selected questions has multiple advantages in the quest for meaningful D&I data.

Response Rate Improvement

Employees are more likely to complete a shorter survey that doesn't demand too much of their time. Longer surveys with numerous questions can be off-putting and lead to higher dropout rates, especially when participants are already busy with their regular workloads. By keeping it concise, organizations increase the likelihood of higher response rates, which leads to more representative and reliable data.

A shorter survey is also more inclusive for individuals with varying attention spans or time constraints, ensuring that all employees have an equal opportunity to contribute their perspectives. This approach aligns with the inclusivity principles that D&I surveys aim to uphold.

One of the benefits of the Divrsity platform is the ability to track (in real time) the amount of time that it is taking our colleagues to complete the survey: it's really powerful to send out an e-mail just before the survey closes saying "on average it takes 4 minutes and 47 seconds to complete a survey". One of our customers recently ran a survey that took nearly 10 minutes to complete, with corresponding drop-off in completion rate.

Increased Focus on Relevant Issues

When surveys are packed with numerous questions, they can feel overwhelming, leading respondents to rush through them or even skip sections. A more focused approach, asking a select few pertinent questions, encourages participants to give deeper consideration to each issue, resulting in richer and more meaningful responses.

By zeroing in on specific topics, organizations can drill down into relevant details, gaining deeper insights into particular areas of interest. This targeted approach allows for a more accurate understanding of the workplace culture, potential barriers to inclusion, and areas necessitating urgent attention.

Data Quality and Actionability

Few things are more detrimental to data quality than diluted or superficial responses. By asking fewer questions, organizations encourage respondents to reflect more deeply, leading to nuanced and thoughtful answers. This enhances the accuracy and quality of the data gathered, making it more valuable for informed decision-making.

When data is precise and actionable, it empowers leaders and D&I teams to implement targeted strategies and initiatives with clarity. A focused survey that delves into a few key topics provides insights that are directly translatable into practical actions and policies.

Constructing Meaningful Questions

So, how should organizations go about selecting the right questions for their D&I survey? Here are some tips to help guide the process:

Start with our Templates

Again, we're biased, but we've got a lot of experience in creating questions that drive meaningful insights.

Clearly Define Survey Objectives

Start by clearly outlining the objectives of your D&I survey. What specific information do you need to gather to gain insights into your organization's culture? Do you want to understand experiences of bias, assess satisfaction with current D&I initiatives, or gauge employees' perceptions of their career advancement opportunities? Setting precise goals helps ensure that the questions crafted are directly linked to measurable outcomes.

Keep It Simple and Accessible

Draft questions that are straightforward and easy to comprehend. Avoid jargon or complex language that might confuse respondents. Remember that some participants may have English as a second language, so keeping the language simple enhances inclusivity. Ensure the questions are also culturally sensitive, avoiding any potential cultural barriers to honest responses.

Focus on Personal Experiences

D&I surveys should prioritize personal experiences to gain individual insights. Questions should encourage respondents to share their lived realities, thoughts, and feelings related to diversity and inclusion in the workplace. For instance, asking about their feeling of belonging, or experience with incidents of discrimination or bias is more impactful than general opinions on company policies.

Maintain Consistency

Ensure your questions are constructed consistently to maintain a coherent survey structure. This aids in comparing responses across demographics and identifying patterns. Consistent phrasing also ensures that respondents aren't confused by differing question formats, keeping the focus on the content of their answers.

Avoid Leading Questions

Be mindful of crafting unbiased questions that don't inadvertently guide respondents toward particular answers. Leading questions can skew results and undermine the integrity of your data. For example, avoid questions that assume certain outcomes or present a biased perspective, no matter how innocuous they may seem. ("Our company is committed to diversity. Do you agree?")

This can be quite nuanced, we recently changed one of our questions that read "When I am working remotely, I feel that I am able to contribute as much as people in the office" because we received feedback that an individual felt that working remotely enabled them to contribute more than people in the office.

Offer Sufficient Context

Provide adequate context for respondents to understand the purpose of the survey and the impact of their responses. A clear introduction should explain the survey's aim, guarantee confidentiality, and emphasize the value of their input. This encourages honest and thoughtful replies, especially on sensitive topics.

We have another article on Comms best practices or you can see our latest questions.

Conclusion

The adage "less is more" rings true when constructing effective EDI Surveys. By choosing a limited number of thoughtful questions, organizations can gather deeper insights, leading to targeted strategies that foster an inclusive workplace culture. Through this approach, D&I surveys become powerful tools for change, rather than mere data-gathering exercises.

Remember, high-quality data is the cornerstone of effective D&I practices, and it begins with a well-designed survey that respects employees' time and voices. The payoff lies in creating an environment where everyone feels valued and heard, contributing to a diverse and inclusive workplace that thrives on rich, actionable insights.