In this study I employed a comparative testing approach to evaluate user preferences and behaviors regarding unavailable payment methods on the public and legacy sites. Two test versions were developed in Lyssna:

 

1. Test Version A: This version presented participants with a checkout scenario where unavailable payment options were grayed out. Participants were shown a screenshot reflecting this design and asked to respond to open-ended questions about their immediate reactions and next steps upon noticing the unavailability of their preferred payment method (E-check).

 

1. Test Version B: In this version, the unavailable payment options were completely removed from the list. Participants were similarly shown a screenshot and asked the same open-ended questions as in Version A.

​

For both tests, participants were presented with the following instructions: 

 

Imagine that you are using our site to purchase a new license plate for your vehicle. Typically, you prefer to pay for important items like this using your bank account (E-check). 

 

These initial instructions were used to simulate the experience of a user checking out on our site and having a desired payment option that would ultimately be withheld from them. These instructions were also intended to align with the screenshots provided by stakeholders who requested this study. 

 

After reviewing the instructions, participants were presented with a screenshot of an unavailable payment option design. 

 

In test Version A, the unavailable payment option was grayed out. In Version B, the unavailable payment option was removed. Each screenshot was presented with the following sentence and two open-ended questions: 

 

When checking out, you notice that the E-check payment option isn’t available. 

• What is your immediate reaction to E-check being made unavailable? 

• After noticing E-check is unavailable, what would be your next step? 

 

After answering the open-ended questions, both test versions asked participants the following multiple-choice question: 

 

In general, when you’re checking out and a payment method isn’t available, would you rather have that method grayed out, or not be shown at all? 

• Don’t show the option at all

• Show the option grayed out 

• No preference 

• Other 

 

This question helped me gain insight into their general preferences outside the context of this study, and was an initial indicator for participant inclinations towards graying out or removing payment options. 

 

Participant responses to the above question provided added context to the final portion of the study which consisted of the following preference test and short answer: 

 

For our site, which design do you prefer: Unavailable payment option grayed out or not shown at all? Please briefly explain your preference.

• Design 1 (Grayed Out) 

• Design 2 (No option Shown) 
   

The selected methodology aimed to eliminate bias by exposing participants to different initial designs and gathering qualitative and quantitative data on user preferences and behaviors.