In my article proving design, I talked about how hard it is to have proofs for making the right product or product decisions. Some projects are so expensive that it takes a lot of convincing to get a budget for them. It’s a natural trade-off. It doesn’t get easier after you’ve done parts of the project, or even after you’ve done the project and are now interested in moving forward with a second stage of development.
Many UX professionals talk about the importance of data but let’s be honest, in the cycle of design and decision making there are countless things that cannot be measured.
What can be measured?
- Do people need your product?
- The product itself and how people use it.
- Ideas, and iterations — Using user research.
Basically everything you can do with your team. It adds up to around 20% of the creation process.
What do you create? What informs your ideas? Are you influenced by other designs you can’t measure? Hell yeah!
What cannot be measured?
Competitors
You can’t know why your competitors behaved the way they did. You don’t have their data and you can’t know their decision-making process.
Pre-product behavior
There are many marketing products that are trying to solve this. However, in this part of the user journey, the designers have zero control. The user journey is driven by the facilitators whether it’s the OS or the platform. Each platform will supply you with some data and measurements but it’s not exactly monitoring UX, it’s more generic and marketing led. In big organizations, it’ll also be a challenge to get these data points. In addition, every piece of data should be verified. With platforms, it’s almost impossible to verify.
Analytics
In your service, you’ll need to check and correlate through different tools (MixPanel, GTM, Data studio, etc.). Understanding analytics tools have become essential for UX and Product roles. This is how companies make crucial decisions and that’s why it is checked and cross matched, usually with three to four systems, to compare and see if the data is reliable.
OS design patterns
The fact that Google decided something should look the way it does doesn’t mean it’s the best way. It means that they probably measured it and it works. It also means that with their level of influence, many apps will adopt it and it’ll become familiar. But it doesn’t necessarily mean it’s better. Some of these decisions are made to differentiate from other platforms like iOS or Windows. Other decisions are a compromised solution to a great design because the design might be patented. That’s precisely why Google and Microsoft bought Motorola and Nokia, stripped them of their patents, and then sold them on to someone else. So if you’ve seen a design, even if it’s famous, it doesn’t mean it’s the best practice.
Just because it works it doesn’t mean it’s a nice experience. Many companies don’t see a reason to change. It’s very common when a company has a monopoly.. For example buying stuff on eBay…does it work?, Yeah…Is it a nice experience? No. Everything is cumbersome: receiving messages, sending, going through versions of eBay from 2000 till today.
In comparison, Amazon are more ambitious and “very slowly” redesign their experiences to be functional and delightful.
You’ve got the Data! But, wait, it might be skewed.
Let’s have a look at how this can happen.
Wrong implementation
Just a simple line of code or a selection of a wrong event could cause every click to be counted as two. That’s why it’s important to check with multiple systems — which, as I mentioned earlier, could be problematic at the stage prior to the user journey beginning in earnest.
Intentions
Even if you have a lot of verified data, how can you believe the data that you see? Every person that collects data (arguably even scientists) is trying to show the data in a way that will flatter their agenda. Data can be collected and presented in a non-neutral way. It’s natural and happens with everyone from marketing companies to UX designers who just want their projects to be successful.
Presentation
The medium is the weapon and it’s important to understand why something was chosen, in a similar way to understanding graphic design decisions: What do they show me, and what don’t they show me.
A few current examples of skewed data and how it has been used:
- Facebook admitted to having wrong measurements for the 10th time
- Facebook is accused of being part of the Fake news problem…Google is too, but it’s used much less for leisure and content consumption.
- Facebook is deleting tens of thousands of Fake users, which is why they keep tweaking the news feed, and Google is doing the same with search results.
- Cambridge Analytica is suspected of and grilled about their methods for influencing users in the UK / US.
Key ways to deal with it
Critic
Be harsh and critical, try to look for the angle. Life sucks if you always think everyone has an interest but even though awareness drives sadness it’s smarter to look at things critically, especially in business. So when you see a new feature, after you finish getting excited or booing it, think about why they created it? Whose decision was it to make it and what’s their interest? Link its value to the business, marketing, user satisfaction, design etc. Guess which department came up with this concept. Think about where they could take it to next. What’s the future of it?
Influenced but aware
There is nothing public, what you have is a trail of user experience data. I got responses for a previous post I wrote about Facebook that said: “But where is the data?” The answer is: This data is internal and not available to anyone else. It’s too secretive to expose, it’s their secret sauce. Does that mean I’m not allowed to write about or analyze it? I don’t think so.
In Instagram, you’d know how many pictures are uploaded to Instagram because it’s a financial data that affects retention/time spent. But you wouldn’t know how many of these pictures are uploaded from a computer, user’s gallery or a professional camera. It’s just important to accept it when critiquing or being influenced by it and to know the limitations of the data you’re dealing with.
Here is an example where you have data, but can only see part of the picturee: “Apple’s revenue from repair is bundled in with its “services” revenue, alongside digital content sales, AppleCare, and Apple Pay revenue. While there is no good way to figure out how much revenue comes from repair, Apple’s services revenue pulled down $7.04 billion in net sales, out of $52.90 billion total.”
Hack
Be aware of your level of control, but see if you can take it further. The difference between owning an OS and participating in one is huge. When I was working for Samsung we were designing the core of Tizen OS for TV. We had control over everything without limitations. We could track everything we wanted to if we built it. But when you are a part of an ecosystem you need to play by the rules and get whatever you can throughout the process. That’s why designing for a native OS is such fun, especially if others are building and increasing your knowledge.
Data is important, but I would argue that decision making can only be done based on it to a certain degree. In my opinion, around 70% of what constructs the decision is experience, aspirations, and alignment with the other sides of the business. A good designer or product guy should influence and convince but it’s not all up to data. Data is just another tool in the arsenal and it’s good for specific use.