Microsoft Research Internship

User’s profile picture is not accessible to the Blind and Low Vision (BLV) users. Investigate mechanisms to support profile picture alt text (Alternative Text) among pervasive digital communication and social media platforms.

This research work enables folks with visual disability to participate in digital representation and provide them the equal access as sighted users to connect with others digitally.

Check out the LinkedIn Redesign prototype for the design deliverable with LinkedIn.

June 2022 - November 2022

Han Feng - Researcher, Designer

Mentor: Martez Mott

Co-Mentor: Ed Cutrell, John Tang, Ann Paradiso

Terminology

Covering the terminologies helps many individuals who are not familiar with the tools used by the Blind and Low Vision (BLV) users. This is a great first step to higher awareness in accessibility and better understand the work space.

Screen Reader:

The software that narrates the contents on the screen for Blind and Low Vision users. Different from the sighted users, they cannot access to the information on the screen visually which requires the software to read aloud for them.

Alt Text (Alternative Text):

Text that screen reader narrates when comes to a picture. Describe the content of picture so that the Blind and Low Vision users can appreciate the images as the sighted users.

Overview & Outcome

Since this applied research project explores a new topic with minimal prior work, the internship was split into two parts:

  • research the foundations and opportunities

  • design an viable interface within a Microsoft product.

This page discusses the research aspects of the internship. Please check out the LinkedIn Redesign page for the design aspects along with the final prototype from research results.

At the end of the internship, I have:

  • Made the LinkedIn prototype to showcase research insights and presented the opportunity for accessible professional connection with tech transfer

  • Recorded two versions result presentations for different audiences to populate awareness of user accessibility and design strategy within Microsoft product teams.

    • 3 minutes version: quick context overview, key element call-out from the LinkedIn prototype

    • 15 minutes version: full details of the research work, walk-through of the prototype with before/after comparison

Literature Review

Although there are increasing amount of research work that surrounds alt text, profile picture alt text is still a new research area with minimal published work. Our goal is to prompt the sighted users to provide high quality alt text that screen reader user desires, which means I needed to explore the topic from various side directions.

The first step of the project is to conduct literature reviews to understand the problem space. This step is to understand what do the screen reader users desire under various context and how should we provide guidelines for high quality alt text. For example, what are elements that screen reader users want when navigating through casual social media platforms, like Facebook or Instagram, versus what do the users prefer in terms of professional connection, like LinkedIn.

With recursive research technique, I have also looked at researches regarding persuasive technology and additional screen reader software interactions. These topics are the supporting justifications to explore optimal profile picture alt text user experience, from both the authoring and consumption standpoint.

Comparative Analysis

 

Based on the literature review results, I have tried to identify the current profile picture alt text situation among popular software and social media services. These products were offer in the version of desktop software, phone app, web pages, or cross offering between multiple platforms.

Result: Out of 31 popular products, none of them offers proper profile picture alt text that could be navigated with screen reader softwares on the profile page (no clicking or secondary menu needed).

Current Situation

Since there is not profile picture alt text functionality, the screen reader will read the profile pictures as:

  • Graphic

  • Graphic, button

  • Han Feng, graphic

  • Han Feng profile picture, graphic

But it does not includes any information about the profile picture that I have carefully chosen: Me standing in front of a beautiful floral background with pink and white roses, wearing a light blue denim jacket and silver necklace, smiling generously to the camera.


“What? There is picture of the person on their contact cards [In Outlook Desktop] ? I have never heard about that through JAWS [a screen reader software], I even do not know how can you access to these images, just like I even do not know the layout or how the Outlook contact cards look like.

— Microsoft Employee with visual disability

Challenge - High Quality Alt Text

The Segue to Design Deliverable

The biggest challenge of this project is not only offering the functionality to provide profile picture alt text, it is more important to provide the high quality alt text that screen reader user desires to know in the profile picture. User wants to know different profile picture elements based on the purpose of the site, such as:

  • Professional Identity (LinkedIn, Work Email)

  • Social Media (Facebook, Instagram)

  • Game (Steam, Xbox)

  • Dating (Tinder, Hinge)

Particularly, it is important to reduce the mismatch between sighted user’s assumption for the alt text elements and the actual desirability from the screen reader users.

Therefore, a guidance or an education mechanism is necessary so that the sighted users are providing accurate and high quality alt text to the screen reader users.

Applied Deliverable

 

The applied deliverable is shown in the LinkedIn Redesign. The requirement for me is to convert the research insights into a design prototype for a Microsoft product that represents one category of website.

I am narrowing the deliverable scope to professional networking sites, particularly LinkedIn. My decision mostly comes from:

  • Professional networking sites play an important role for meeting recruiters, connecting business networks that met in-person, and other professional networks that fosters career development

  • Professional identity is usually more prioritized by the users as they want to show their “best self”, and the profile picture alt text can further populate the contents

  • LinkedIn is the industry leader in professional networking websites, and the opportunity to be an industry-first website to support profile picture alt-text is highly impactful and profitable.

Feel free to read more details at the LinkedIn Redesign.

Looking Forward

 

This is only the first step to kick off the profile picture alt text work, and there is a long journey ahead of us to make these research result more mature. Especially, how to apply this alt text practice to all digital platforms involving profile pictures is a good challenge to consider.

So far, I have a few potential iterations / ideas to flush out:

  • User Narrated Alt Text: Sighted users read out their profile picture alt text through an audio clip to reinforce the identity representation and the authenticity of a real person author to the screen reader users.

  • Varying Detail Alt Text: Screen reader users hear two versions of alt text with different level of details. If the users are searching people from the search bar, they will hear the low-level alt text from the auto-fill list. The full version alt text will be available on a user’s primary home page.

  • Profile Picture Element Alt Text: Within LinkedIn, users can indicate if they are hiring or read to work. There can be an alt text to incorporate these special elements.