Why do most online discussion packages suck so badly?

Submitted by brad on Tue, 2007-04-17 18:36
Topic: 
Internet
Tags: 
usenet

Yesterday I attended the online community session of Web2Open, a barcamp-like meeting going on within Tim O'Reilly's Web 2.0 Expo. (The Expo has a huge number of attendees, it's doing very well.)

I put forward a number of questions I've been considering for later posts, but one I want to make here is this: Where has the innovation been in online discussion software? Why are most message boards and blog comment systems so hard to use?

I know this is true because huge numbers of people are still using USENET, and not just for downloading binaries. USENET hasn't seen much technical innovation since the 80s. As such, it's aging, but it shouldn't be simply aging, it should have been superseded long ago. We've gone through a period of tremendous online innovation in the last few decades, unlike any in history. Other old systems, like the Well, continue to exist and even keep paying customers in spite of minimal innovation. This is like gopher beating Firefox, or a CD Walkman being superior in some ways to an iPod. It's crazy. (The users aren't crazy, it's the fact that their choice is right that's crazy.) One reason USENET has not evolved is that nobody owns it, and as such, there is less motivation for people to fix and improve the parts they own and control. Individual web sites have owners, and those owners are keen to make them better to either attract more attention or make more money. This has moved most innovation into the web space.

USENET is still superior to most web messaging systems at dealing with high volume traffic. The traditional USENET arrangement involved local access to material that was pre-fed to a local server. As such, access to articles was local and instantaneous. Once users taste instantaneous access, it is very hard to go back to slow remote response times, even for fancy new features. (Though AJAX applications like GMail, which do some things with local access speed, have managed to pull people away from local apps into remote ones.) More recently, RSS readers have been able to pre-fetch information to provide an instantaneous UI to material published that way, which may finally deliver this functionality. Today, many people access USENET through a more remote server, as many ISPs simple contract out USENET service to an outside company, though they try to get a quick, low-latency connection to it.

Likewise, USENET delivered everything in one application. In a typical USENET application, users track all their subscriptions to all newsgroups, and get the same interface to all of them. This is not present in web message boards, but is done for RSS readers. RSS readers, however, are primarily used with news feeds and blogs. Many sites do allow them for comments but their use is not very common for threaded message boards, and indeed most do not present a threaded interface.

The threaded interface, which came to USENET with Wayne Davison's 'trn' over 20 years ago is found in a number of web message boards. However, a variety of basic functions on threads are sometimes found, but more rare than should be expected:

  • Understanding which articles have been "seen" (not necessarily read) and which have not, and presenting threads in this fashion
  • Quick skipping to the next thread (single-keystroke) or elimination of a branch of a thread or an entire thread from reading in both a current session, and permanently.
  • Permanent skipping of posts by particular users (known as a killfile or bozo-filter.)
  • Browse, choose and read: USENET readers allow users to examine the set of threads with new messages, quickly pick the threads they would like to read, and then present as quickly as possible just the articles in selected threads, with easy skip onward.

USENET at its peak had newsgroups with several hundred thousand readers involved and many seeing 10,000 or more messages per month. Yet it was and is possible to follow an online community of that size using the USENET tools. There are sites today with message volumes on single topics in these ranges but they are usually browsed and read scattershot. USENET handled about 4 million articles/month in 1995, in an internet vastly smaller than today's.

It is often felt that online community simply doesn't scale, and that's true when it's close-knit. There, the limit seems to be somewhere between 100 and 200 intimate users. However, better tools can present ways to have workable communities that are less closely knit, but still have commonality at larger sizes. However, I don't see that much of them.

Of course, I have not tried every possible web message board out there. Some are much better than others, and I welcome input on what you think are the best, and why.

One factor that may play into this innovation problem: advertising. Commercial web sites now almost all are funded by advertising. Making it very quick and easy to get through all the messages on a web site may not appear to be in an ad-seller's interest, though it should be if it attracts more users. Of course, I didn't write the blogging software I am using (drupal) so it has just the same basic threaded comment system you see everywhere, though it does at least offer rss feeds of comments and comments on particular posts. However, it's currently not very easy to track comments on particular threads around the web with RSS feeds. For one thing, the polling approach of RSS (vs. USENET's and E-mail's "push" approach) makes it hard to deal with a thread that dies out slowly, and few RSS readers are set up to have you be subscribed to thousands of threads, only a few of them being updated, and do it in an efficient way and with a threaded presentation.

Share on:
  • Reddit logo
  • Facebook logo
  • Twitter logo
  • LinkedIn logo

Comments

Phillip Helbig

Wed, 2007-04-18 07:43

I think you've hit the nail on the head. I have spent about a dozen hours a day at a computer, and probably half
of that somewhere on the internet, for the past 15 years. I have a computer job, have computers at home where
everything is run locally, have access to the latest hardware and software, am technically curious. I am also using
the same software for email and news that I used 15 years ago. Why? As you say, it does things better. I think
the commercial aspect is a big one. The internet wasn't commercial in the past. Today, large parts of it are.
Efficient software and advertising just don't go together.

Probably one of the worst innovations is a web-based interface to newsgroups or, worse, discussion groups which
are entirely web-based, with no usenet backend. True, in the old days, the distributed approach was necessary
due to limited bandwidth etc. The main point isn't that, though, but is the interface. I like to subscribe to
a group, see only unseen messages, set a message, an entire thread or an entire group to seen at a keystroke. I
can do that with a newsreader. I can't do that with a web-based interface. As you say, another advantage is the
same interface to all groups, not the case with all the forums out there.

I also use mail and news readers with a character-cell interface. At home, I have a nice terminal or a nice
terminal emulator. However, at almost any internet cafe in the world, I can fire up a DOS fenster, connect to
my home system, and read mail or news with my familiar software. With the most common commands mapped to single
keystrokes, it's very efficient and I don't need a mouse.

By the way, I also have to enter your last name in lower case only to get accepted. Bug or feature?

alex goldman

Wed, 2007-04-18 10:44

I think too many people are trying to re-invent the wheel. The wheel, IMO, is dailykos.com. Now, this is not a live discussion software, but it has many web 2.0 features.

My question is why aren't people imitating it?

Features:

1) Access all of your comments by date
2) View everyone who recommended your comments
3) AJAX-based refresh makes a hot thread feel active as new comments arrive every 60 seconds
4) Open and close any comment (slashdot feature missing: view only recommended comments)
. . . and more

I'm sure you can think of other apps that have done one thing very well, but have had those innovations ignored.

(snark: maybe we need a microsoft, a company that will take everybody else's innovations and put them into a single software package. I used to like a word processor called Nisus Writer and I miss some features on the Leading Edge Word Processor.)

brad

Mon, 2007-05-07 17:32

Or at least that's what it says.

DailyKos does seem to consist of mostly very short comments. This is in contrast with USENET and a number of mailing lists that tended to focus on fewer, longer comments. I am not sure which is better. (I guess I would like to have well thought out comments distilled to their essence.)

Jack

Mon, 2007-05-07 17:15

I think DailyKos has outgrown it's site also. It's getting slightly harder to use the more information that is posted.

Anon Y. Mouse

Wed, 2007-05-09 13:41

I can remember when most Usenet traffic was carried by UUCP,
so this "instantaneous access" seems relatively recent to me.

But yes, web-based discussion boards do suck in comparison to
Usenet tools like trn or Agent. I have seen one good feature
on this board or another on that one, but never have I seen a
single message board that comes close to having it all. I
consider it good fortune if my browser history can track which
messages I've read, but often this breaks down because the
link changes as posts are moved onto other pages or into
archives. I think there are several factors that contribute
to this deficiency in web message systems.

Perhaps the biggest is the stateless nature of HTML. Consider
some of the features you mention: group subscription, thread
watch or elimination, read/unread status, killfiles. Usenet
readers track all this with meta-information stored locally on
your hard drive. Encoding all that information in a generated
link is out of the question, so a web discussion board has to
have users identify themselves via login and then store all
their related data on a server -- which is where the problems
start.

Of course all the data *can* kept on the back-end, but that
makes coding a web board much more work. In the first place,
all the necessary tracking data has to be created, used, and
stored. The data also has to be transformed and carried
forward as new features are developed, or else users would
become annoyed and angered as their state information was lost
with each new version of the software. As noted,
site-specific message boards may be constrained by commercial
interests. Between those limitations and programmer laziness,
there is little motivation to do all the work necessary to
create the features that have become standard in the Usenet
world.

Another factor is the broader audience towards which the
typical web-based board is aimed. I would say the average
Usenet participant is more technically savvy than the average
web surfer. So I suspect that the average web board user is
less likely to understand and use, much less appreciate, the
more sophisticated discussion management features.

Finally, I think it's very possible that many web board
designers and programmers have never used Usenet, or even
heard of it. Consequently, they have never seen an example of
what's possible and have very low expectations about what
constitutes a complete discussion package.

Greg

Fri, 2009-01-16 18:56

Brad, I think you are totally correct. I grew up on Usenet and can tell you that I still can not get totally comfortable with all the new fangled methods of web communication. But I also cringe when I see how young some of these guys are who invent and make billions (see Mark Zuckerberg). I have two nephews who are both in college at an ACC school (in computer related degrees) and neither had ever heard of Usenet! One is a graduating senior and the other is a junior. I am afraid it is going to take something groundbreaking to return Usenet to the minds of our young.

Add new comment