Comparative Usability Study
The purpose of this comparative usability study was to evaluate discoverability of key features on the TV UI when pressing the Power and Home buttons on the TV remote.
Background and Research Goal: Dating back to 2016, a menu bar of streaming apps (e.g., Netflix, Prime Video, etc.) would appear along the bottom of the Smart TV UI upon pressing the “Power On” and “Home” buttons on the remote. In 2018, a new browsing feature was developed that would live below the menu bar of streaming apps. This new feature was intended to support browsing by surfacing content “up-front” through aggregating rows of TV shows and movies from various streaming apps. With the integration of this new “content-first” browsing feature to our traditional “app-forward” menu bar, questions surfaced how to map the “Power On” and “Home” buttons on the remote and whether defaulting to either feature would impact discoverability of the alternative key feature. The main goal of this research was to identify the impact, if any, to the navigability and discoverability of each key feature when the “Power On” and “Home” buttons on the TV remote defaulted to the opposing state of the UI.
My Contributions: I led all project planning phases including: Designed the methodology; drafted the study plan and schedule; secured and managed participant recruitment vender; independently drafted the discussion guide; coordinated prototyping efforts; independently moderated sessions; led debriefing sessions with team who actively observed sessions; analyzed data and generated actionable insights; and packaged insights into deliverable which was shared out to all stakeholders.
Method: Two prototypes were generated by our prototype team, one where the focus state of the Power and Home buttons mapped to the (traditional) menu bar of streaming apps, and one where the focus state of the Power and Home buttons mapped to the new “content-first” browsing experience. A Within Subjects Design was used with 8 people for 90minute one-on-one sessions, where participants were assigned to alternating conditions of the prototype for the first half of their session in order to balance learning and order effects. Initial discovery of each key feature was noted. A Wizard-of-Oz technique was used to facilitate swapping the version of the prototype, mid-way through participant sessions while Participants took a short water break. Upon return of their break, navigability and discoverability of each key feature was noted while participant explored the new prototype version assigned for their second half of the study. In post-task concluding questions, each participant was exposed to both conditions, one after another, and asked for their preference and why.
Outcomes: It was found that initial discoverability of the menu of apps was high regardless of the prototype design that participants experienced first; people largely cited familiar with streaming service (e.g., Netflix) icons as to why. On the contrary, initial discoverability of “content-first” experience (below the fold) suffered when the focus state of the Power and Home buttons on the remote defaulted in menu of streaming apps.
Impact: As a result of this research, the menu bar of apps remained the focus default state and design brainstormed new visual affordances to include on the UI that now encourage greater discoverability of the new “content-first” experience.
Two Usability Labs exist at the Samsung Mountain View campus, one designed to replicate a living room and the other to replicate a studio:
