[2021] New question: disabilities & accessibility

[SachaG]

One thing I'd like to start doing in 2021 is focusing more on accessibility and disability issues. For example, asking people if they have permanent or temporary disabilities could help highlight that these issues are more common than one might think, and push other developers reading the survey results to give them more thought.

My current thinking is to have two multi-choice questions, one for permanent disabilities and one for temporary ones.

Permanent Disabilities

Are you affected by these conditions?

Temporary Disabilities

Are you currently, or have you been in the past, affected by one or more of these conditions?

List of Disabilities

• Blindness
• Partially impaired vision
• Inability to type
• Inability to walk
• Inability to hear
• ...?

Questions

• Is it better to phrase the options in terms of condition ("carpal tunnel syndrome") or effect ("inability to type")?
• Should we try to be as granular as possible ("color blindness", "near-sightedness") or more general ("impaired vision")?
• Is there a list somewhere of disabilities?
• How do other surveys handle this?
• Are there offensive terms we should avoid?
• Should we also ask about coping strategies? (screen readers, etc.)
• What about non-physical disabilities? Depression, etc. ? Are those even "disabilities"?

[laurakalbag]

I think it’s important to consider whether you are asking these questions for demographic information, or for information about how people use the web, and avoid mixing the two. (You do not need specifics on people’s disabilities for either.)

Before you go any further, I would read as much as you can about the language around disability and improve this issue. Compiling a list of offensive terms to avoid is… a bit weird. There’s a lot of (mostly US-centric) resources on the conscious style guide which recommends a resource from National Center on Disability and Journalism which covers some good areas. There’s a little more UK-English-centric advice on the UKs’s government site. These are all resources that come to mind quickly, there are plenty more to be found.

WebAIM has a long history of doing surveys in the area of web accessibility. I would recommend following their work and reading up on the many blog posts (by them and others) that reflects on the results of these surveys and the effectiveness of the questions.

And above all, find and pay an experienced consultant (preferably a disabled person) to help you do this properly. Honestly, I struggled with your survey last time because I found the section on race exclusive and pretty offensive, you aren’t going to get representative results if people stop doing the survey because they feel excluded by the questions.

[SachaG]

@laurakalbag thanks so much for your feedback!

I think it’s important to consider whether you are asking these questions for demographic information, or for information about how people use the web, and avoid mixing the two.

Could you elaborate on this? I'm not sure I understand the difference? Do you mean that instead of asking people if they have condition X, I should ask if they have resulting disability Y instead?

And above all, find and pay an experienced consultant (preferably a disabled person) to help you do this properly.

Sadly this won't be possible for a while, as the project is not really profit-driven at the moment. I do think there's value in some initiatives being more community-led, so hopefully we can work around this limitation and still do a good job on this topic!

Honestly, I struggled with your survey last time because I found the section on race exclusive and pretty offensive, you aren’t going to get representative results if people stop doing the survey because they feel excluded by the questions.

I don't think our goal is to not offend anybody, because the mere fact of asking questions about race is enough to offend people who are often uncomfortable with dealing with these issues. And I also suspect the phenomenon of "this offends me, I quit!" is much more prevalent among non-minorities ("white fragility" and all that…). But if you have specific issues with the phrasing then I'd love to hear more!

[laurakalbag]

Could you elaborate on this? I'm not sure I understand the difference? Do you mean that instead of asking people if they have condition X, I should ask if they have resulting disability Y instead?

For demographic information, you probably don’t need to ask anyone anything more than "do you identify as disabled?" What does the survey gain from knowing a person’s specific disability unless it can do anything with that information?

Accessibility is where it’s more useful to know about a person’s disability. But again, you don’t need to know what a person’s disability is… every person experiences their disability in massively differing ways. (And bear in mind that many people who don’t necessarily identify as disabled have impairments that affect their use of the web… for example people who need glasses to read text on screens.)

(I assume) what you want to know is how disability affects how somebody experiences the web, so the questions should focus on that. (WebAIM’s surveys cover this to some degree with a focus on use of screenreaders, and it would be worth discussing with at least a few disabled people whether they feel accurately represented by those questions.)

I don't think our goal is to not offend anybody, because the mere fact of asking questions about race is enough to offend people who are often uncomfortable with dealing with these issues.

Asking about race isn’t the issue (it’s arguably important), it’s that the racial segments you chose excluded many groups of people, conflated country and race, and made a weird distinction (for me as a mixed-race person) between bi-racial and multi-racial that made me wonder "does it make a difference to this survey whether my heritage is that I’m mixed between two races or more than two races?!"

My point being that, if you don’t ask these questions in a way that you can get meaningful answers, then you are better off not asking the question at all, as, at best, your results will be misleading, at worst, the quality of the survey’s results will be called into question.

Sadly this won't be possible for a while, as the project is not really profit-driven at the moment. I do think there's value in some initiatives being more community-led, so hopefully we can work around this limitation and still do a good job on this topic!

People from marginalised groups are less likely to have time/ be able to afford to participate in community-led projects, and are more likely to face discrimination or abuse as a result of participating in these projects participated in by mostly cis/straight/white/non-disabled people. You can get some value from well-researched community members who are trying to do the right thing, but you will always be more effective if you go directly to people who have experience and first-hand knowledge in the area. If you get a basic idea of what you want organised, it may only take a few paid hours to have an experienced consultant iterate on it, and will improve your survey dramatically. You could even find out how much it’d cost, and crowdfund for the fees. There are many people who care about inclusivity in our industry who would be willing to contribute a few euro to having survey results that they can use to inform their working decisions.

[SachaG]

(I assume) what you want to know is how disability affects how somebody experiences the web, so the questions should focus on that. (WebAIM’s surveys cover this to some degree with a focus on use of screenreaders, and it would be worth discussing with at least a few disabled people whether they feel accurately represented by those questions.)

Yes exactly, the intent is not to know the specific disability but the way it impacts their use of the web. So for example if someone broke their arm we don't need to know what happened, but we do want to know they have impaired mobility that might make it hard to use a mouse, for example.

it’s that the racial segments you chose excluded many groups of people, conflated country and race, and made a weird distinction (for me as a mixed-race person) between bi-racial and multi-racial that made me wonder

We had a separate discussion thread about that question, but just to briefly address your points, I got the list of ethnicities from Stack Overflow's survey. It's true I don't know why they make a distinction between biracial and multiracial, so since I can't defend it I'll probably drop it from future surveys.

As for conflating countries and race, I get your point but I suspect that's just a compromise they came up with to try and cover all the different ways people identify themselves? And about missing segments, the main one that got reported was South-East Asian, we'll add it next year.

If you get a basic idea of what you want organised, it may only take a few paid hours to have an experienced consultant iterate on it, and will improve your survey dramatically.

That's a good point, I guess even a few hours of someone's time would be helpful and we can definitely afford that. I'll have to think about it, but if you have anybody in particular to recommend let me know.

[cferdinandi]

I don't think our goal is to not offend anybody, because the mere fact of asking questions about race is enough to offend people who are often uncomfortable with dealing with these issues.

cringe. What a phenomenally bad take.

[MelSumner]

Here are some potential question ideas for this topic:

1. Do you have a disability that affects your relationship with technology?
2. If so, rate the available tools to support you as a developer.
3. Where do you go as a developer to find out how to make your code accessible?
4. How satisfied are you with the accessibility information provided in the documentation for your preferred framework?
5. What could your preferred frameworks do to make it easier for you to write accessible code?

I think there are a lot of questions you could ask about accessibility from the perspective of accessibility as an engineering practice.

I do agree with @laurakalbag's advice to seek the advice of a tech person who is disabled for best results. A consultation could at least set you on the best path for this line of questions.

Anyway, just some ideas, good luck!

[SachaG]

cringe. What a phenomenally bad take.

It was probably poorly worded, so let me elaborate. Obviously our goal is never to go out of our way to offend people, nor do we want to ignore people who say they've been offended. My point was more that if you adopt "try not to offend people" as your guiding principle, you risk ending up where tech has been collectively for the past decade, which is trying to avoid dealing with matters of gender, race, etc. in order not to ruffle any feathers. If we went that route, we would simply not ask these questions at all, and I would probably sleep much better at night!

But I do think the survey can have a positive role to play in this debate, which is why I've made my peace with the idea that I might get things wrong in the process of educating myself and offend people; or even get everything "right" (for a given definition of "right") and still offend people, just because it's such a complex and multi-faceted topic.

[SachaG]

@MelSumner thanks for your great suggestions! I totally agree with seeking the advice of people who experience these issues themselves on a daily basis.

[SachaG]

OK, so based on everything that's been said I'd like to suggest the following questions as a starting point (which will of course be submitted to an expert for review):

1. Disability Types

Which of the following disabilities do you currently have, or have experienced in the past? Check all that apply.

• Vision impairment (Blindness, low vision, etc.)
• Color-blindness
• Psychiatric disability (Autistic Spectrum Disorder, Depression, etc.)
• Cognitive impairment (Brain injury, brain fog, intellectual disability, etc.)
• Deafness/Hard-of-Hearing
• Upper-body motor impairment (Carpal Tunnel, broken arm, etc.)
• Lower-body motor impairment (leg injury, etc.)
• Whole-body motor impairment (Parkinson's, etc.)
• Other (freeform text field)

(Note: not sure if it makes sense to differentiate between upper/lower/whole-body impairments? Also not sure where color-blindness fits in)

Sources used (in addition to those already mentioned in this thread):

• https://services.anu.edu.au/human-resources/respect-inclusion/different-types-of-disabilities
• http://disabilityinfosa.co.za/mobility-impairments/types-mobility-impairment/
• https://www.health.ny.gov/diseases/conditions/vision_and_eye_health/types_of_vision_problems.htm
• https://en.wikipedia.org/wiki/Visual_impairment

2. Accessibility Tools & Strategies

Which of the following tools or strategies do you regularly use online as way of coping with your own disabilities?

• Screen reader
• Dictation/Speech-to-text
• Text-to-speech
• Closed-captions/audio descriptions
• Increasing screen magnification or text size
• Alternative/ergonomic keyboard and/or mouse
• Keyboard navigation
• Braille keyboard/display
• Sip/puff switch
• Hands-free mouse control (eye-tracking, face tracking, etc.)
• Other (freeform text field)

Sources:

• https://www.w3.org/WAI/people-use-web/tools-techniques/
• https://monsido.com/tools-web-accessibility-for-disability

3. Accessibility Features

Which of the following accessibility features do you regularly implement in your own work?

• Keyboard shortcuts
• Tab-based navigation
• Alt/title attributes, alternative text
• Visible focus indicator (:outline)
• Valid and semantic markup (avoiding divs when more specific HTML elements exist)
• ARIA landmarks & markup
• "Skip to main content" links
• Dark mode or high-contrast mode
• Prefers-reduced motion media query
• Other (freeform text field)

Sources:

• https://www.w3.org/WAI/people-use-web/tools-techniques/
• https://webaim.org/projects/motordisabilitysurvey
• https://www.a11yproject.com/checklist/
• https://www.washington.edu/accessibility/checklist/

4. Accessibility Testing

Which of the following tools do you use to check that your own work meets accessibility standards?

• Contrast checker
• Screen reader
• Visual impairment simulator
• Accessibility checker or evaluation tool (Wave, etc.)
• Accessibility checklist
• Other (freeform text field)

Sources:

• https://wave.webaim.org/

---

Other Notes

• I'm not quite sure yet if we'd ask these questions in every survey we do (we are hoping to eventually expand beyond State of JS and State of CSS with State of GraphQL, State of Node, etc.) or maybe just State of JS and/or State of CSS. But in any case I think they would be a good starting point.
• We always try to feature checkbox questions as opposed to open-ended fields just because it's much harder to process the results of freeform text fields.
• I'm conscious of the huge selection bias created by the fact that our own accessibility failings will directly impact the answers we get (i.e. if our survey platform doesn't work with screen readers, then the number of people who answer that they use screen readers will be very low…). Does that mean we should wait until we fix our own accessibility issues before we ask some of these questions?
• And of course, this is very much still a work in progress so feedback welcome on any of this!