Lucene for Go

github.com/malcolmston/lucene

Embedded full-text search for Go, in the style of Apache Lucene.

 GitHubports apache/lucene

A small, dependency-free, in-memory full-text search engine written in pure Go, modelled on Apache Lucene. A configurable analysis pipeline tokenizes, lowercases, drops stop words and stems your text; an inverted index records term frequencies and positions for every field; and a rich query model — with a query-string parser — is ranked by BM25 into deterministic top-N hits. Everything is built on the Go standard library alone: no cgo, no third-party modules, nothing to audit but the toolchain.

Install

shell
$ go get github.com/malcolmston/lucene

Quick start

main.go
import "github.com/malcolmston/lucene"

idx := lucene.NewIndex(lucene.NewStandardAnalyzer())
_ = idx.Add(lucene.Document{ID: "1", Fields: map[string]string{
	"title": "The Go Programming Language",
	"body":  "Go is an open source programming language.",
}})

// Parse a query string and take the top 10 hits by BM25 score.
res, _ := idx.SearchString("body:programming +body:go -rust", 10)
fmt.Println("matches:", res.Total)
for _, hit := range res.Hits {
	fmt.Printf("  %s  %.3f
", hit.ID, hit.Score)
}

Features

  • Analysis pipeline — NewStandardAnalyzer tokenizes, lowercases, drops stop words and stems, tunable via WithStopWords and WithStemming
  • Inverted index — NewIndex with Add / Delete over Document values; postings carry term frequencies and positions, and it is safe for concurrent use
  • Query model — TermQuery, PhraseQuery, BooleanQuery (Must/Should/MustNot), PrefixQuery, RangeQuery and MatchAllQuery
  • Query-string parser — NewParser and Parse turn title:go "phrase" +must -not net* [a TO z] into a query tree
  • BM25 relevance — Search returns a Result of top-ranked Hit values, ties broken by document ID for fully deterministic output
  • One-call search — SearchString parses and executes a query string against the index's analyzer in a single step
  • Highlighting — NewHighlighter and Highlight wrap matched (and stemmed) terms in custom markers while preserving the original text
  • Zero dependencies — pure Go standard library, no cgo, no third-party modules
Lucene