Unity Test on the public

[christopher.wortman]

I took a System76 laptop into the public with Ubuntu 11.04 and Unity to get reactions to the OS. Here are the results, I might travel to a different area and do a video review of it. When I presented it to them, I presented it as a brand new product, that I was doing market research to see if this would sell.

I tested this on around 200 people
76% of the people had an almost immediate response of being lost and not knowing what to do. They couldn't figure out how to use global menu, in some applications and how in LibreOffice it was in the application. They said it was "confusing". Actually:

83% had the immediate response that it was "confusing".

A small percentage said it reminded them of a cheap Mac knock off, and that it should research "copyright infringement". That they owned Macs at home and that Apple should see this.

About 20% had a non-plus reaction to it. It seemed to work fine, but why reinvent the wheel?

Almost nobody compared it to Windows, there were like 3 people who did. They are the type of people who wouldn't own a cellphone because landlines would suffice. Obviously not the intended audience for this.

While the responses were not entirely negative, people had an oddly positive response. Even the ones who thought it was "confusing" seemed to like it, but said it needed work.

"The menu up top should not disappear" seemed to be a ressurecting response to the global menu. They liked the side bar, but wanted to move it to the right as it felt out of place on the left, or the icons should auto-sort to the right for sake of keeping things neat and tidy.

When we interviewed the people who said it was confusing, they responded with how the bar was anchored to the left, the bottom/left would be a better place for it. The menu keeps dissappearing. The full screen menu, and scrolling through makes no sense for it to be that big. They wanted the icons to be smaller. They likened it to a cellphone OS on a computer.

Almost nobody became frustrated, however, and seemed to call it fun and inventive. That it was "a joy to use". The fact that it is still "in market research, some of the annoyances will presumably be fixed."

This was very informative and tought me a lot. Since this is driven for new users, keep up the good work I look forward to testing this on 11.10. I have a little more faith in those who developed Unity, and I had fun with this. I kinda can't wait to do this again.

[el_koraco]

The fact that it is still "in market research, some of the annoyances will presumably be fixed."


lol, that's sweet.

[Dry Lips]

My brother finally decided to install Ubuntu after I had been
telling him that he ought to give Linux a try. He also confessed
that he had trouble finding his way around, and seemed happy
when I showed him how to boot into the classic mode.

I'm no big fan of the shell of Gnome 3 either, but I think it is a
lot more structured than Unity.

[TheSuperSteve]

Good read. Canonical or whoever the big bosses over Ubuntu happen to be should see this. Or at least do this kind of research on their own, if they haven't already.

I always think that an OS should be tested with our parents/grand parents. If they can figure out how to use it or if figuring out how to use it is intuitive enough for them to not need help, then the thing is ready.

Of course this being the Linux community, such an OS as described above would be sinful. As if some people feel good about themselves if they know how to compile programs from the terminal or something. Its a computer, the simpler it is, the better. That's how we go forward. We make things simpler and faster. Of course, i do agree that much of Unity is different for the sake of being different. Its also buggy and slow in its current state. Which is the reason why i dont take too kindly to it. Not because i dont like new things.

Eh, i'm kind of going on a rant/ramble. Proceed with your thread. I enjoyed the results, such as how a small percentage called it Mac-like. That's the reason why i like Unity. XD

[Triblaze]

Very interesting results, more research like this should be done.


Maybe next time you could do a comparison? Maybe show them that and then either the default GNOME or KDE environment, just to see which design elements work best and which don't work.

[akand074]

These results are hardly interesting. It's a pretty obvious result. Every single Windows user there just tried to use it like it was Windows... and it wasn't the same. Obviously you should find it confusing when you first use it and you are used to something else. Takes time to get used to it and learn about it's pros/cons. When I first started using Ubuntu, I only had it on my hard drive but only logged onto it on occasion to play with compiz, and I generally knew what I was doing when it came to gnome 2 panels. It's not until I got so fed up with Windows that I booted into Ubuntu and swore never to go back to Windows on my desktop again that I used it for a little while and started really getting used to it and really liked it.

Point is, you can't judge it unless you have each person use it strictly for say a month and then get their response. Anyone generally who hasn't used any OS and then walked into Windows would be really confused and likely not enjoy it.

That said, Unity really needs work and is not ready for a large audience. 11.10 should be great but by the next LTS I'd expect it'd be better to get a better view of what people think. I think if Ubuntu became large enough that major manufacturers supported it seriously, that they would only install LTS by default.

[Copper Bezel]

These results are hardly interesting.
Indeed; intending no offense to the OP, what results? A majority of respondents were confused but oddly positive. I don't really know what that means. Oh, and 76% couldn't figure out how to make an operating system work when it was shoved in their faces on the street.

I think the information provided would be more useful if accompanied by the survey questions used and a little description of how these interactions went. Again, no offense, but 90% of respondents tend to support a ban on dihydrogen monoxide, when it's phrased the right way. = )

[tumbes2000]

Everyone is confused when they see something new. The first time I used Mac os 9 I was really confused coming from win 95/98. Yet I ended up liking it. Basically the same thing played out with Mac os x and anytime I switched oses (red hat linux for example back in the day). The good thing is that people ended up feeling positive about unity.

If apple can make it so my mother can use a unix in only one day, Ubuntu can do the same with linux and seems to be headed in that direction.

[Perfect Storm]

Good read. Canonical or whoever the big bosses over Ubuntu happen to be should see this. Or at least do this kind of research on their own, if they haven't already.


They have.

[Khakilang]

Recently I have converted 1 of my brother's friend when I show him my Linux. He seem happy about it and dual boot with Vista. The last time I heard he hardly touch Vista and plan to totally wipe out Vista and use 11.04. He seem to be comfortable with Unity and he is totally noob about computer. Unity is not at all that bad even for new user.

[CraigPaleo]

If someone came up to me and asked me to try out an OS, I wouldn't tell them it sucks even if I thought it did. I'd probably tell them what I thought was wrong, what I thought was right, and leave some encouraging words such as "it's not for me but it has potential."

When people start learning to use a computer for the first time, they most likely don't know there's an alternative. They learn it no matter how confusing it may be. The problem is that most people learn on Windows or Macs. Once they realize there are other choices, most of them expect it to work and feel the same as what they've been using.

They may try any flavor of Linux but since it's free, they're likely to go back to what's familiar if it seems to different. If those same people bought a Mac, they have a pretty good incentive to learn it and get used to it. The alternative being to pay for a copy of Windows.

It's like me trying to get back to learning QWERTY. It's just too easy, when I get fed up, to switch back to Dvorak. If I had switched to QWERTY and then had to pay to go back to Dvorak, I'd probably be typing in QWERTY right now.

[dinamic1]

If someone came up to me and asked me to try out an OS, I wouldn't tell them it sucks even if I thought it did. I'd probably tell them what I thought was wrong, what I thought was right, and leave some encouraging words such as "it's not for me but it has potential."

When people start learning to use a computer for the first time, they most likely don't know there's an alternative. They learn it no matter how confusing it may be. The problem is that most people learn on Windows or Macs. Once they realize there are other choices, most of them expect it to work and feel the same as what they've been using.

They may try any flavor of Linux but since it's free, they're likely to go back to what's familiar if it seems to different. If those same people bought a Mac, they have a pretty good incentive to learn it and get used to it. The alternative being to pay for a copy of Windows.

It's like me trying to get back to learning QWERTY. It's just too easy, when I get fed up, to switch back to Dvorak. If I had switched to QWERTY and then had to pay to go back to Dvorak, I'd probably be typing in QWERTY right now.

nah, unity sucks

[Thewhistlingwind]

If apple can make it so my mother can use a unix in only one day, Ubuntu can do the same with linux and seems to be headed in that direction.

Not just any Unix. BSD! (And mach, and.....just forget it....) If you've ever tried using BSD, you'd know that what apple did with it is nothing short of amazing.

I think however, that the kind of OS and utilities that most people want, isn't the OS and utilities that I would want.

If I made a DE it'd be based around keyboard shortcuts, for example.

[BFG]

The problem with unity is you have to invest heavily in use of the interface. This eventually builds strong community feelings as you become "one of the club of people who get it". Hence the vehement enthusiasm from some quarters.

Maybe this is deliberate, but it flies in the face of the modern technology successes where systems are immediately intuitive and anyone can pick something up and perform magic immediately.

If deliberate, Unity is not good enough. Too much is broken and there's not enough choice. A thousand papercuts and lots of assumption that a user conforms to a limited configuration. Run a dual-head setup? That mouse journey from your app bottom-right to the <snip> top menu is a papercut all right - across your eyeballs.

[Copper Bezel]

The problem with unity is you have to invest heavily in use of the interface. This eventually builds strong community feelings as you become "one of the club of people who get it". Hence the vehement enthusiasm from some quarters.
Nothing about Unity requires any more investment than any other UI.


If deliberate, Unity is not good enough. Too much is broken and there's not enough choice. A thousand papercuts and lots of assumption that a user conforms to a limited configuration. Run a dual-head setup? That mouse journey from your app bottom-right to the Mac-kissass top menu is a papercut all right - across your eyeballs.
I don't think almost anyone would claim that Unity is in a final, stable form right now, which is what the "thousand papercuts" process refers to, the hard work of getting the details working. And the menu panel should be displaying on your second monitor.

[Grenage]

I tested this on around 200 people
76% of the people had an almost immediate response of being lost and not knowing what to do.

...

83% had the immediate response that it was "confusing".

I'm fairly switched on; when I used a Mac for the first time, I found it confusing. A 30 second glance at an OS is not a fair test.

[RoflHaxBbq]

I hate ignorant people who compare it to Mac/Windows. Apparently if you have a bar of any type at all at the top, you're copying OSX. If you have a bar at the bottom, you're copying Windows. Gnome has a bar at the top, but it's functionality is completely different to OSX's bar. In OSX applications are launched from the bottom and controlled from the top. In Gnome, applications are launched from the top and controlled from the bottom. What gives?

[CraigPaleo]

I'm fairly switched on; when I used a Mac for the first time, I found it confusing. A 30 second glance at an OS is not a fair test.

Thirty seconds is all it takes for most people to decide whether they like it or not. These are the same people who haven't so much as changed their wallpaper in two years. Only the more curious ones will take the time to figure everything out.

We had only Macs in schools here. That's what I knew and that's what I bought. My dad had Windows 3.1 and I couldn't understand how anyone could use it. Then I had a friend tell me that this new Win 95 was just like a Mac. It was NOTHING like a Mac! It took me about 30 seconds to figure that out. At the time, that's all I needed to know.

Eventually, I did learn how to use Windows but that was out of necessity. Linux and open source, on the other hand, piqued my curiosity. I learned it because I was interested in it - not because it was forced on me. I've grown to love the flexibility of Linux and its various DEs. I can configure my machine almost any way I want it. With Unity, you cannot even move the dock or top panel easily if at all. Gnome Shell isn't much better. They're worse than Mac, which I'll never go back to, where you can at least move the dock.

Let those same people try Gnome panel or KDE and I bet the results would be much more favorable. They are both much more intuitive. They'd probably last longer than 30 seconds with them.

[Thewhistlingwind]

With Unity, you cannot even move the dock or top panel easily if at all. Gnome Shell isn't much better. They're worse than Mac, which I'll never go back to, where you can at least move the dock.


Using the gnome-shell right now, I'd agree, part of the advantage of the Linux DE's was their flexibility both in implementation and their relative separation from the kernel. Any Linux distro can, in theory, use any DE.

I hope these DE's gain back their customization over time, or they'll likely lose a large portion of their users to XFCE and KDE.

[CraigPaleo]

Using the gnome-shell right now, I'd agree, part of the advantage of the Linux DE's was their flexibility both in implementation and their relative separation from the kernel. Any Linux distro can, in theory, use any DE.

I hope these DE's gain back their customization over time, or they'll likely lose a large portion of their users to XFCE and KDE.

Gnome 3 fallback is still customizable. You can configure the panels but you have to Alt right-click on them (non-intuitive). It's buggy though. Sometimes the Alt right-click would work; other times, nothing would happen. It's lacking themes and applets but they should come in time.

I don't think Gnome shell and Unity were meant to be very configurable, which is great for supporting them but not so good for those of us who want things just so.

[Thewhistlingwind]

I don't think Gnome shell and Unity were meant to be very configurable, which is great for supporting them but not so good for those of us who want things just so.

Let me be more specific. "I hope the Gnome and Unity teams realize that they are currently making a horrible design decision, may in fact be mentally insane, and really should give their users what they want."

I have no illusions about the chances of this happening of course.

EDIT:




Either way, this doesn't need to be another 'I hate Unity' thread.

Of course not, I'm pretty sure the Unity devs know that by this point.

That was quite possibly some of the most powerful backlash I have ever seen.

[Grenage]

Let me be more specific. "I hope the Gnome and Unity teams realize that they are currently making a horrible design decision, may in fact be mentally insane, and really should give their users what they want."

I have no illusions about the chances of this happening of course.

I think that long term, it will probably be a very good desktop. While it's 'yet another' system, at least this one has a big backer and might actually get somewhere.

Either way, this doesn't need to be another 'I hate Unity' thread.

[CraigPaleo]

Let me be more specific. "I hope the Gnome and Unity teams realize that they are currently making a horrible design decision, may in fact be mentally insane, and really should give their users what they want."

I have no illusions about the chances of this happening of course.

I agree. New users may be familiar with Android or iOS but would they want it on their personal computers?

[CraigPaleo]

I think that long term, it will probably be a very good desktop. While it's 'yet another' system, at least this one has a big backer and might actually get somewhere.

Which is why it should listen to its customers.


Either way, this doesn't need to be another 'I hate Unity' thread.

I don't hate Unity. I just think it's a bad design decision.

[Bazon]

Good read. Canonical or whoever the big bosses over Ubuntu happen to be should see this. Or at least do this kind of research on their own, if they haven't already.


They have.

Yes, they have, but in a ostrich-like manner.
I explain with one concrete example: In the april report (http://design.canonical.com/2011/04/unity-benchmark-usability-april-2011/), there is written:

Multi-tasking: Having many items opened and accessing them

October report: “Thus, while working on a task, participants expected that Unity would provide them with a representation or visibility of what was available to them and how to easily access what they needed at any given point.”
April testing: No problems were observed with overlapping open applications and documents. Participants could easily move individual windows and reveal items placed underneath.
Outcome: This is fixed.

So I simply ask: What had changed in Unity from October2010 to April2011 to explain this? As far as I can see (I also tried Unity in the netbook edition..):
Nothing changed in the design of Unity related on this, only the question was asked in another way in order to better cover the actual problems:
"No problems were observed with overlapping open applications and documents. Participants could easily move individual windows and reveal items placed underneath."
uh-huuu, moving windows isn't a goal that high to achieve, is it?

But of course, reducing the Multi-task question to moving windows and so avoids covering problems like that in the October report (http://design.canonical.com/2010/11/usability-testing-of-unity/) again:

Multitasking – Multitasking on Unity is disconnected and difficult at times

Task flow is interrupted: While working on a task, participants wanted to have all the documents and websites they were using easily available to them. For them, the task was the unit of organisation of all resources and tools. Thus, while working on a task, participants expected that Unity would provide them with a representation or visibility of what was available to them and how to easily access what they needed at any given point. Unity does not, however, make evident the resources and tools users have at their disposal — whether it be multiple documents, programmes or websites. In a word, while Unity relies on users to keep track of the resources they are currently using, users are habituated to relying on the software to “keep track for them,” by making the resources highly visible, for example, by means of tabs. With Unity, resources are hidden from view.

Overall, participants found the navigation between documents to be cumbersome.
(...)

And that goes on for several interesting paragraphs, read it if you want to (http://design.canonical.com/2010/11/usability-testing-of-unity/).


Seriously, that is a bad scientific approach, that's hiding the head in the sand in ostrich-like manner. That doesn't lead to make anything better, and having the impression Canonical deals problems that way makes me sad. :-(

[tjeremiah]

My mom is a windows user. She would always use windows at home and uses it daily at work. At first she was confused when I showed her Unity but 2 months later, she starts the OS up on her own and knows how to navigate the system without having to call me.

Heck, I know someone here said they were first confused by a Mac. I was too. The majority of the computers in my college uses them and I was lost when first using it. I quickly ran to find the nearest Windows OS in order to get some work done.

Also, enough of the UNITY crying already. Im sure the developers hear loud and clear people's displeasure of the OS. No need for another thread where someone once again points out how Unity is more of a miss to people.

[BFG]

I don't think almost anyone would claim that Unity is in a final, stable form right now, which is what the "thousand papercuts" process refers to, the hard work of getting the details working.

That process was a hundred papercuts, "thousand" is an ironic reference to how far off it is. ;)

[Copper Bezel]

Really? Thousand is far more poetic, and I could have sworn that was the original, I.e. not in reference to software. Apparently "1000 papercuts" is another thing entirely.

CraigPaleo, you are arguing that people who don't change the default wallpaper on their Windows machines will want to move the Unity panels. Explain.

[CraigPaleo]

CraigPaleo, you are arguing that people who don't change the default wallpaper on their Windows machines will want to move the Unity panels. Explain.

No. I was just trying to say that people who are set in their ways aren't likely to spend a lot of time trying out an unfamiliar OS unless they have a reason to.

[cgroza]

No. I was just trying to say that people who are set in their ways aren't likely to spend a lot of time trying out an unfamiliar OS unless they have a reason to.
It took me no more than 5 minutes to figure out how Unity works.
And if they are trying Unity, it is because they have time to spare.