Ideas and Plans for Improving Ubuntu Documentation

[Pragmatist]

You could do it with AJAX or just server side (boring). ;P

Even though I was talking about using a"bot", maybe we don't need that much versatility.

I'm think about a client-side program to reduce bandwidth. The program (for instance, for the purposes of discussion, let's say a java applet) would just gather information, search several databases, and return an organized response. This would have at least two advantages:

1.) Gives relevant information to the potential poster(related, solved, posts; howtos/guides; etc...) These would be returned to the screen in categories. One box containing links to the Howtos, another with links to related, solved posts, and so on.

2.) Gets relevant information from the user. Depending on the nature of the problem, certain standard information would be requested. For instance, if it were a hardware problem, the user would be asked to add the output of lspci and dmesg and lsmod and so on, to their post.

Both of these don't really require a "bot".

Info from user
Getting the information from the user is trivial; Basic flowchart logic gives a series of finite problem types. Each problem type has a corresponding set of information to be requested from the user.

Info to user
Returning the information is also not that hard. After being asked a series of questions (another flowchart) our program has a bunch of keywords (a bit more than that since our program gets information in response to specific questions, our program has a correlation between the two. This correlation allows the program to search more intelligently--such as what sites to search at, for instance).

One of the key open questions is: What are the different technologies currently used by the different Ubuntu websites/tools? Database, Search feature, server side programs, client-side programs, are the different sites connected in some ways? How hard would it be to have a unified search feature for the different websites?

On this last question, if there were a unified search feature, and there were plenty of meta-data on the different sites, then this would improve things dramatically. For instance, instead of using seperate search engines, our program would just use one. When the data is returned, our program analyses the meta-data to extract basic "type" information such as "is this a howto" "is this a post" "is this a recommendation" etc..etc.. This will allow our program to return an organized reponse to the user.

Any ideas on how we can find out what website/webserver technologies are being used on the Ubuntu sites?

[az]

Even though I was talking about using a"bot", maybe we don't need that much versatility.

I'm think about a client-side program to reduce bandwidth. The program (for instance, for the purposes of discussion, let's say a java applet) would just gather information, search several databases, and return an organized response.

yeesh! Java doesn't ship with ubuntu! But I see your point. The idea is technology-agnostic.

This would have at least two advantages:

1.) Gives relevant information to the potential poster(related, solved, posts; howtos/guides; etc...) These would be returned to the screen in categories. One box containing links to the Howtos, another with links to related, solved posts, and so on.

2.) Gets relevant information from the user. Depending on the nature of the problem, certain standard information would be requested. For instance, if it were a hardware problem, the user would be asked to add the output of lspci and dmesg and lsmod and so on, to their post.

Both of these don't really require a "bot".

Nor do they require the forums. The forums are too volatile for such a knowledgebase tool. The wiki is too static to be perfect. Anyway, this is a docteam issue. You are talking about Ubuntu documentation at large, not just the posts on the forums. It would best serve the cause to discuss it with the Doc Team.

Info from user
Getting the information from the user is trivial; Basic flowchart logic gives a series of finite problem types. Each problem type has a corresponding set of information to be requested from the user.

Info to user
Returning the information is also not that hard. After being asked a series of questions (another flowchart) our program has a bunch of keywords (a bit more than that since our program gets information in response to specific questions, our program has a correlation between the two. This correlation allows the program to search more intelligently--such as what sites to search at, for instance).

One of the key open questions is: What are the different technologies currently used by the different Ubuntu websites/tools? Database, Search feature, server side programs, client-side programs, are the different sites connected in some ways? How hard would it be to have a unified search feature for the different websites??

In the early days, the docteam had quite a number of discussion regarding such technologies. The problem was that the wiki was supported by Canonical and didn't meet every need perfectly, but it met many more of the needs that other technologies.

The fact that due to security vulnerabilities, James Troup wanted nothing to do with php4 on his servers complicated things. This is also part of the reason why Canonical did not host forums at the beginning.

At the time, the docteam were limited to either using technologies other than php for their documentation portal or find outside hosting. Since php5 is out and the Fridge runs on Drupal (which uses php), I guess this is sorted out. However, the wiki and the docteam mailing list is working out acceptably.

It would be interesting to revisit this. Maybe an interactive yelp?

On this last question, if there were a unified search feature, and there were plenty of meta-data on the different sites, then this would improve things dramatically. For instance, instead of using seperate search engines, our program would just use one. When the data is returned, our program analyses the meta-data to extract basic "type" information such as "is this a howto" "is this a post" "is this a recommendation" etc..etc.. This will allow our program to return an organized reponse to the user.

Any ideas on how we can find out what website/webserver technologies are being used on the Ubuntu sites?

The wiki runs on Moin. The mailing lists are about to be ported over to launchpad, if I understand correctly. There was supposed to be a launchpad authentification plugin for php that would log you into the wiki and any other launchpad-enabled Ubuntu site automatically. Maybe UbuntuGeek can fill you in on how that is going.

Jon Dodson said he could write it, if needed and the last time I checked, the offer still stands.

It would really rock if the forums could offer a mailing list section for the docteam mailing list. A lot of forums users would probably participate in the docteam that way. The forums are probably more relevant to the docteam than is currently exploited.

[Pragmatist]

Nor do they require the forums. The forums are too volatile for such a knowledgebase tool. The wiki is too static to be perfect. Anyway, this is a docteam issue. You are talking about Ubuntu documentation at large, not just the posts on the forums. It would best serve the cause to discuss it with the Doc Team.

Thank you for your comments. I didn't think of the documentation team. Your totally right that this is too specific for discussion here. However, regarding this part of your statement:

The forums are too volatile for such a knowledgebase tool.

I'm not sure what you mean exactly. The idea I'm thinkin of is to just make the tool available, not force it. For instance, anytime somebody starts a thread, one of links they can press would be labeled "try our---fill in the blank---tool!). Then a smaller link explaining how the tool works. If nobody click's either link, then it is never used. Just like the normal forum search feature. You don't have to use it...but its there.

[az]

I'm not sure what you mean exactly. The idea I'm thinkin of is to just make the tool available, not force it. For instance, anytime somebody starts a thread, one of links they can press would be labeled "try our---fill in the blank---tool!). Then a smaller link explaining how the tool works. If nobody click's either link, then it is never used. Just like the normal forum search feature. You don't have to use it...but its there.

What I mean is that a forum is best suited for discussion. The end result would be for the user to find proper documentation, right? In the forums, there can be several threads about the same topic. That's great for documentation in the larger sense, but the end-result would be for the user to be pointed to a specific text or ressource, and not neccessarily the discussions pertaining to those texts (or which led to the creation of those texts).

Maybe sometimes the user would need a coarser-grained search tool, but you seem to describe something that incorporates that, but is not limited to that.

Like Scotty says, "the right tool for the right job."

[Pragmatist]

I think we are in agreement about the usefulness of the kind of tool I describe. We are not completely in agreement about where it belongs, however.

Originally Posted by azz
What I mean is that a forum is best suited for discussion.

Yes and no. By definition a forum is a place for discussion. However, a "help forum" is different. While there certainly can be different opinions on how to install a specific hardware device there are only so many right answers (answers that solve the problem). Please correct me, but my guess is that most of the threads in the beginner forum, if not several others, involve finding solutions to problems. This is different than, say, a forum about ancient chinese philosophy, or opinions on what GUI works best for a forum. Those have more room for opinion and discussion. So forums that focus more on problem solving would benefit the most from a tool such as I describe.

Originally Posted by azz
The end result would be for the user to find proper documentation, right?

My point is that the end result of many of the Ubuntu resources is to solve a problem (and hopefully educate as well...see my previous posts). To that extent, I think it is unnecessary, at best, to create a division between problem solving sources. The tool I suggested would help to unify these sources, and, in doing so, improve all of them.

Originally Posted by azz
...but the end-result would be for the user to be pointed to a specific text or ressource, and not neccessarily the discussions pertaining to those texts (or which led to the creation of those texts).

It is true that forum threads inform documentation creation. However, it is also true that documentation informs forum thread creation. The better the documentation, the less threads, the less posts in threads, the less frustrated OPs, etc.. The two go hand-in-hand. The better they are integrated, the more efficient they both become.

[majikstreet]

the bot thing sounds like something that microsoft has

[az]

I think we are in agreement about the usefulness of the kind of tool I describe. We are not completely in agreement about where it belongs, however.



Yes and no. By definition a forum is a place for discussion. However, a "help forum" is different. While there certainly can be different opinions on how to install a specific hardware device there are only so many right answers (answers that solve the problem). Please correct me, but my guess is that most of the threads in the beginner forum, if not several others, involve finding solutions to problems. This is different than, say, a forum about ancient chinese philosophy, or opinions on what GUI works best for a forum. Those have more room for opinion and discussion. So forums that focus more on problem solving would benefit the most from a tool such as I describe.



My point is that the end result of many of the Ubuntu resources is to solve a problem (and hopefully educate as well...see my previous posts). To that extent, I think it is unnecessary, at best, to create a division between problem solving sources. The tool I suggested would help to unify these sources, and, in doing so, improve all of them.



It is true that forum threads inform documentation creation. However, it is also true that documentation informs forum thread creation. The better the documentation, the less threads, the less posts in threads, the less frustrated OPs, etc.. The two go hand-in-hand. The better they are integrated, the more efficient they both become.

I think we agree, but make different assumptions. Yes, a tool which integrates all the different sources of documentation would be best. I am just assuming that the end-user will get more mileage out of a tried-and-true piece of documentation that gets updated periodically than from the thread-of-the-day on the topic. As such, I guess I see this as a tool which incorporates the forums, but is not strictly a "forum tool".

But yeah, don't give me just the one-liner solution, and don't give me the ten pages of discussion about it from people who may or may not know the answer. Give it *all* to me.

Don't get me wrong. I will ***** myself to any degree to make the forums more integrated into the greater ubuntu community and vice versa (at the risk of being banned, actually). I think this is something that can help that. In that respect (and in my opinion), I think it has been discussed before and if you try to provide the forums as a be-all and end-all source of documentation, a couple of bad posts lead to people saying that it is crap, and no one wants to see that happen.

[az]

the bot thing sounds like something that microsoft has

How well does it work and how can we do it better?

[KiwiNZ]

to make the forums more integrated into the greater ubuntu community and vice versa (at the risk of being banned, actually). I think this is something that can help that. In that respect (and in my opinion), I think it has been discussed before and if you try to provide the forums as a be-all and end-all source of documentation, a couple of bad posts lead to people saying that it is crap, and no one wants to see that happen.

Can you clarify this a tad

Thanks

[az]

Can you clarify this a tad

Thanks

This thread:
https://lists.ubuntu.com/archives/ub...il/001771.html
and:
https://lists.ubuntu.com/archives/ub...il/001753.html


I don't have all the conversations archived in my mailbox, but you really should ask Corey or anyone else from the docteam. You must have been absent (the tsunami) when comments were made about the forums content being "absolute crack".

That is a long time past, and the docteam are much closer to the forums now, but the point is that if you regard the forums as being a singular source of documentation, one good or bad post can make or break the reputation.

I have to reiterate that the people to talk to regarding this are the docteam people.

To be absolutely clear, I also have to say that I think the relationship between the docteam and the forums is a lot stronger now than ever before.