December 29, 2016

Code indexing with lein-codeindex

I just pushed lein-codeindex, a brand new Leiningen plugin for easy code indexing and referencing.

My last big project involved a lot of Clojure code and I got tired of building tags file every time when someone adds a new library or manually hunting for dependencies project is using. In short, I was a little bit envy on Cursive my colleagues were using, simply because Cursive (and other IDEs) would pull the code dependencies and try to index and reference as much as possible.

In Emacs-land there aren't many options - you'd either have to invoke etags manually, fiddle with shell scripts to figure out dependent libraries, hunt for custom regexes as etags does not recognize Clojure or use lein-gentags, which scans only the code present in given project folder.

Or, there is object metadata and :file and :line keys, where is stored symbol location. To see what I'm talking about, this sample shows location of println referenced in your REPL:

user=> (def src-path (comp str clojure.java.io/resource :file))
user=> (src-path (meta #'println))

;; this output depends on clojure version, system, jars load order and etc.
jar:file:/home/user/.m2/repository/org/clojure/clojure/1.8.0/clojure-1.8.0.jar!/clojure/core.clj

These are limiting options; beside already mentioned etags/lein-gentags usage, object metadata will return code of current symbol, but you can't rely on tag completion or listing similar symbols without complex machinery.

So, lein-codeindex is my humble attempt to improve this situation. Even better, you don't have to use Emacs to use it; Vi/Vim, Sublime Text and users of other editors can benefit from it too.

Quickly skim over README to see how to install it as I'll cover common usage here.

In your project folder, run:

$ lein codeindex

This will create Emacs tags file (named TAGS) with references to the project code and all external dependencies. Want to know how into works? Navigate to symbol and hit M-. or M-x tags-search into in case you are using Emacs.

Emacs etags isn't perfect; if you'd like to use ctags, which is fast and has flexible language mappings, make sure you have it installed (usually comes with Vim) and run:

$ lein codeindex --ctags

This will generate Emacs tags with ctags engine. You'd like to use it with Vim? Run:

$ lein codeindex --vim

or

$ lein codeindex --ctags --vim

and you will get tags file, usable from Vi/Vim (and other editors I believe). lein-codeindex has other options and you can see them with:

$ lein help codeindex

One notable option is –no-langmap, available only when –ctags option is set. ctags allows setting custom languages and custom regular expressions for recognizing symbols in $HOME/.ctags and if you have one like this or similar for Clojure, use this option to skip builtin language maps in lein-codeindex.

How it works

It's not rocket science :) lein-codeindex relies on Leiningen API to get all dependencies and proper jar locations in local maven repository; this is crucial option, because it will select only those libraries currently used by the project where is invoked.

Then, it will simply unpack it in .lein-codeindex folder and scan for Clojure sources (files with extensions .clj, .cljs, .cljc and .edn), invoking etags or ctags on them. You can further customize folder name by setting LEIN_CODEINDEX_DIR environment variable, but make sure it is stored in your project folder.

Extracting jars will add also unnecessary files, like META-INF folder or class files, but since this is the first release, I'll keep this as feature request in one of the next versions.

Conclusion

Try it, use it and if something brakes, let me know.

Tags: clojure