DrBacchus' Journal

In general, I would agree. I do, however, think that there is a call for docs aimed at developers rather than end users...particularly early in the life of a software project. I do, of course, agree that projects need good docs for users, particularly as they get to "release quality" (whatever that may mean). Ultimately, I think most software projects need docs for both developers and users.

Oh, and I do think a partial retraction of your defense of ASF docs is in order. Yes, httpd's docs are good (great, in fact), but as you're seeing, many of the docs for other projects are sad, sad, sad. I share the experience of looking through many Apache projects and not even being able to figure out what the project does. "Annoying" doesn't describe it.

Jeff

It varies by project. I too found lucene a bit difficult to follow. There were some articles scattered around that helped clear things up but you certainly shouldn't be required to read external articles understand any particular project. I believe there is a book coming out at some stage.

Lucene is, at it's core, a Java library (API) for indexing and search that doesn't have any other dependencies at all. There are extensions that may require containers, but it all depends on what you want to do with it. In any case, the community mailing list is very active and helpful.

S

Well, I was hoping to set up a simple search engine on a simple (all static) existing web site. That was all. I will get on the mailing list and see how newbie questions are received. I really don't mean to gripe, and would be glad to contribute to a documentation effort in any way that I can.

What is wrong with reading Lucene articles?
(lame comment entry - doesn't allow links)

Here's another aspect of Open Source that annoys the non-initiate. When they complain about how hard it is to find useful information for beginners, they get told that they are not working hard enough, and that if it's hard, it's because they are lazy (or stupid) and not because the docs are terrible.

Your criticism is like complaining about a garage that sells car engines when you were really looking for a hire car. There's a market and need for both, with typically different clients.

Lucene only sets out to be a library - not a server or end-to-end search solution. Having limited boundaries for open-source projects is desirable and makes it a lot easier to combine different libraries into custom solutions.

If you're looking for a server you can run "out of the box" - commercial solutions such as SearchBlox package the Lucene library with document parsers, admin consoles and documentation.

Actually, no, it's more like a garage that sells engines but doesn't have a clear sign that tells me what it sells. Thanks for clarifying what Lucene is, and what it is not.

I have this same complaint with a significant number of the newer Apache projects - I have absolutely no idea what they are, and there's no clear statement anywhere as to what they are. And so I waste a great deal of time trying to get things to work which end up not being what I thought they were.

I do indeed appreciate you stating what it is. I would appreciate even more a clear statement on the website saying what it is.

A while back I thought it might be a good idea to go through all the Apache projects and try to state what each one is. Unfortunately, I never found the tuits. I'll put that back on my ToDo list.

First sentence on Lucene homepage.
"Jakarta Lucene is a high-performance, full-featured text search engine library written entirely in Java."

I don't know what else you expect it to be. I think it's very clear. It's a LIBRARY, not a complete software. Although it comes with some command line interfaces, it is intended to be a LIBRARY.

holyman,

I'm going to make the (probably incorrect) assumption that you're asking a real question, that you genuinely want to know why people don't understand what it says, and that the Lucene people care about making it understandable.

What else would I expect? I would expect that sensible people would not develop a library in a vacuum. That sensible people would develop an implementation, however small and limited, which would exercise the API of a library that they were building. And that, at the very least, the web site would point to such implementations.

I would further expect that a good percentage of visitors to the site would overlook the word "library" in that sentence, and see "high-performance, full-featured text search engine." A large part of writing good documentation is to understand how beginners see your sentences, and how they will understand them.

What I see very consistently is that people inside the community know what it does, and those outside it don't. That's the mark of documentation that is written with a very limited audience in mine. Yes, developer-centric documentation is important for small, immature projects, but as those projects mature, it's important to aim it more towards people stumbling on the project for the first time.

Of course, if the question was intended merely to be argumentative and belligerent, you probably didn't read this far at all. But, I read your message loud and clear: negative feedback not welcomed, go elsewhere for your needs. This is a shame, coming from a project with in the ASF, but I suppose the character of the ASF is inexorably changing, and I just need to get used to it.

funny coincidence: an old friend asked me yesterday about lucene (since he knows I do java and do java @ the ASF, he assumed I'd at least know something...which I really don't). This guy was also looking for a ht://dig replacement, which lucene is not.

There's several projects with this kind of problem. It all has to do with audience...this friend of mine was not the target audience for lucene, nor are you. Many "middleware" or backend projects have this kind of problem. Geronimo is indeed another good example. It isn't directed towards "new" users at all right now. Rather, the initial target audience is people who have several yeras of experience with server-side java programming.

You see this a lot, not just in open source, and we all experience it from time to time: you're interested in something, but there's "prerequisite reading" before you'll be able to grok all of it.

You can't go to oracle.com and learn what a database is. Its assumed you know what a database is, and why you need one, before you ever visit the site. The microsoft windows homepage doesn't even hint at the fact that windows is an operating system. The page, to a casual non-windows-aware person, makes windows sound like some kind of security software (!).

The obvious thing to do is for the authors of documentation and software to describe their "target audience", and "prerequisite reading".

The HTTPD documentation is great if you're the target audience (namely, someone who wants to set up a webserver). Not if you want to learn what a webserver is, let alone what a server is (isn't a server a hardware device that makes a lot of noise?). Not if you want to learn how to write a webserver.

HTTPD omits the "target audience" and "prerequisite reading" materials, as do windows, oracle and countless other projects.

We could all learn a little from how the OSAF is managing chandler publicity.

Rich - I've been keenly following this entry and the comments since you posted it. As a huge fan of Lucene, an author on a couple of java.net articles on Lucene, a frequent speaker on the topic of Lucene, and finally a co-author of the upcoming Lucene in Action book (Manning) I want to see Lucene shown in the best possible light. Your comment is taken to heart and there is no doubt that all of us developing open source software could do a better job at documenting and promoting our projects. Lucene is no exception, and its web site could definitely use some clarifications on what it really is.

I often "spelunk" into various open source Java projects myself and find the documentation and explanations unclear. I'm so immersed in the Java world, though, that it feels natural to me to simply skip the documentation and go right to some example usage and then further into the source code itself when I need more clues.

While I do concur that Lucene's web site could use more (and I will do what I can to enhance it in the next couple of months), like another poster said about the first sentence, I think it provides a pretty clear and succinct description of Lucene. For a web crawler built on top of Lucene, give Nutch (http://www.nutch.org/docs/en/) a try. It has recently been modified to facilitate intranet indexing/searching more easily.

If you have any questions on Lucene, I'd be more than happy to hear from you via e-mail.

Erik