#+TITLE: treesit-auto
#+AUTHOR: Robb Enzmann

#+html: <a href="https://www.gnu.org/software/emacs/"><img alt="GNU Emacs" src="https://github.com/minad/corfu/blob/screenshots/emacs.svg?raw=true"/></a><a href="https://melpa.org/#/treesit-auto"><img alt="Melpa" src="https://melpa.org/packages/treesit-auto-badge.svg?raw=true"></a><a href="https://stable.melpa.org/#/treesit-auto"><img alt="Melpa" src="https://stable.melpa.org/packages/treesit-auto-badge.svg?raw=true"></a>

Automatically install and use tree-sitter major modes in Emacs 29+.  If the
tree-sitter version can't be used, fall back to the original major mode.

* Features
Each of these behaviors are configurable and documented under the
"Configuration" section.  By activating =global-treesit-auto-mode=, Emacs will:

+ Automatically switch to =<name>-ts-mode= when the grammar for =<name>= is installed
+ Stick with =<name>-mode= if the grammar isn't installed
+ Automatically install a grammar before opening a compatible file
+ Modify =auto-mode-alist= for tree-sitter modes

There is also a convenience function =M-x treesit-auto-install-all=, which will
install all (or a selected subset) of the maintained and compatible
grammars. You can add these grammars to your =auto-mode-alist= automatically by
invoking the =treesit-auto-add-to-auto-mode-alist= function in your
configuration.

* Installation
~treesit-auto~ is available from [[https://melpa.org/][MELPA]].  After following [[https://melpa.org/#/getting-started][their setup]], you can
use your preferred package manager.  If that's the default =package.el=, simply
=M-x package-refresh-contents= and then

#+begin_example
M-x package-install RET treesit-auto
#+end_example

If you want a local clone of the repository, rather than just a copy of the
source, you might instead use =package-vc-install=

#+begin_src example
M-x package-vc-install RET https://github.com/renzmann/treesit-auto.git
#+end_src

Then, in your Emacs configuration file (=~/.emacs.d/init.el=),

#+begin_src emacs-lisp
  (use-package treesit-auto
    :config
    (global-treesit-auto-mode))
#+end_src

For most users, this will be enough.  There are some nifty things you might want
to enable, though, which are covered in the "Configuration" section below.

* Configuration example
This is how I configure =treesit-auto= for my own personal use.

#+begin_src emacs-lisp
  (use-package treesit-auto
    :custom
    (treesit-auto-install 'prompt)
    :config
    (treesit-auto-add-to-auto-mode-alist 'all)
    (global-treesit-auto-mode))
#+end_src

* What this package does
Emacs 29, while featuring =treesit.el= and a convenient
=treesit-install-language-grammar=, [[https://archive.casouri.cc/note/2023/tree-sitter-in-emacs-29/index.html][will not feature an intelligent way to choose]]
between a default mode, such as =python-mode=, and its tree-sitter enhanced
version, =python-ts-mode=.  This package attempts to remedy that by adjusting the
=major-mode-remap-alist= and =treesit-language-source-alist= variables in order to
get the following behavior:

*1. If the grammar is installed, then switch to the appropriate tree-sitter mode:*

In this case, assuming we open a Python buffer, and the [[https://github.com/tree-sitter/tree-sitter-python][Python tree-sitter
grammar]] is installed, then Emacs will use =python-ts-mode= instead of
=python-mode=.

*2. The grammar is NOT installed and treesit-auto-install is non-nil:*

When the grammar is not installed and ~treesit-auto-install~ is t, then upon
activating any major mode that has a corresponding tree-sitter mode, the grammar
will be downloaded and compiled using ~treesit-install-language-grammar~.  Emacs
will then activate the tree-sitter major mode for that buffer.

~prompt~ is like t, except a message will be displayed in the echo area asking
for a yes/no response before attempting the installation.

As an example for both cases: if I visit a Python file and didn't already have
the grammar installed, I wind up with an installed grammar and a buffer using
~python-ts-mode~.

Otherwise, when ~treesit-auto-install~ is nil, it will try to fall back to
another major mode as described in the following two rules.

*3. If the grammar is NOT installed, and a fallback is specified*

Most languages will have a fallback mode specified, such as =python-ts-mode=
falling back to =python-mode=, if the grammar is not installed.  If you ever need
to double-check what that fallback will be, you can double check what's in the
recipe for that language like this:

#+begin_example
(treesit-auto-recipe-remap (alist-get 'python treesit-auto-lang-recipe-alist))
    ⇒ python-mode
#+end_example

See "Configuration/Configuring behavior for a specific language" in case you
would like to specify different fallback modes than the default.

*4. The grammar is installed, but there is no fallback mode*

You can optionally use the =treesit-auto-add-to-auto-mode-alist= function to
ensure that your installed tree-sitter languages have their corresponding
=...-ts-mode= added to =auto-mode-alist=, so that Emacs opens the buffer in that
=...-ts-mode=, rather than the default =Fundamental= mode.

Supposing for instance we don't have =typescript-mode= installed, then even if
we /do/ have =typescript-ts-mode= installed along with the typescript grammar
compiled in =~/.emacs.d/tree-sitter/=, Emacs still won't use
=typescript-ts-mode= unless you also added ='("\\.ts\\'" . typescript-ts-mode)=
to =auto-mode-alist=.  By calling =(treesit-auto-add-to-auto-mode-alist)= in
your configuration, this modification to =auto-mode-alist= is done automatically
for you.

*5. All other cases...*

This is the most general case, where the grammar is not installed,
~treesit-auto-install~ is nil, and no fallback mode is specified in the language
recipe present on =treesit-auto-recipe-list=.  In this case, we still gain the
benefit of quickly installing grammars through =treesit-install-language-grammar=
without having the build the recipe interactively, but =treesit-auto= will make no
attempt to switch away from the tree-sitter mode.

* Configuration
If you have modified =treesit-language-source-alist= through =setq=, then it is
recommended to put any configuration of this package AFTER that =setq=.

** Choose which languages =treesit-auto= should consider

You can globally alter the behavior of =treesit-auto= to only consider a
specific set of languages by setting the =treesit-auto-langs= list to a set of
language symbols.  By default, this list includes every possible language that
=treesit-auto= supports, so you can use =M-x describe-variable RET
treesit-auto-langs= to see what the options are.

One way to use this variable is to just set it manually:

#+begin_src emacs-lisp
  (setq treesit-auto-langs '(python rust go))
#+end_src

Now, =treesit-auto= features will only ever affect Python, Rust, and Go files.
Running =treesit-auto-install-all= will only install these three grammars, and
no automatic prompting/installation will occur when visiting a buffer that is
not one of these three, either.

Another method is to disable specific languages by just removing them from this
list:

#+begin_src emacs-lisp
  (delete 'awk treesit-auto-langs)
#+end_src

Here, =treesit-auto= behaves as it normally would for all languages /except/ AWK.

** Automatically install grammars if they are missing
The =treesit-auto-install= variable controls whether a grammar should be installed
automatically when activating a major mode compatible with tree-sitter.

1. =nil=, the default, means =treesit-auto= won't try to install anything, and will
   rely on the fallback logic outlined above
2. =t= means =treesit-auto= should always try to clone and install a grammar when
   missing
3. ~prompt~ will cause a yes/no prompt to appear in the minibuffer before
   attempting installation

#+begin_src emacs-lisp
  (setq treesit-auto-install 'prompt)
#+end_src

Then, supposing I don't have =libtree-sitter-python.so= (or its mac/Windows
equivalent) under =~/.emacs.d/tree-sitter= (or anywhere else in
=treesit-extra-load-path=), visiting a Python file or calling =M-x python-ts-mode=
will generate this prompt:

#+begin_example
  Tree-sitter grammar for python is missing.  Would you like to install it from https://github.com/tree-sitter/tree-sitter-python? (y or n)
#+end_example

Responding with "yes" will use =treesit-install-language-grammar= to go fetch and
compile the missing grammar.

The other function that respects this variable is =treesit-auto-install-all=.
When =treesit-auto-install= is t, using =M-x treesit-auto-install-all= will skip all
prompts.  Otherwise, it will ask before attempting the installation.

** Configuring behavior for a specific language
The variable =treesit-auto-recipe-list= keeps track of all the language "recipes."
These control how =treesit-auto= decides which modes to upgrade/downgrade to/from,
where the source code of the language grammar is hosted, and which C/C++
compiler to use.  Each recipe can take these arguments:

#+begin_example
:lang
:ts-mode
:remap
:url
:revision
:requires
:source-dir
:cc
:c++
#+end_example

To create a recipe, use =make-treesit-auto-recipe=:

#+begin_src elisp
  (setq my-js-tsauto-config
        (make-treesit-auto-recipe
         :lang 'javascript
         :ts-mode 'js-ts-mode
         :remap '(js2-mode js-mode javascript-mode)
         :url "https://github.com/tree-sitter/tree-sitter-javascript"
         :revision "master"
         :source-dir "src"
         :ext "\\.js\\'"))

  (add-to-list 'treesit-auto-recipe-list my-js-tsauto-config)
#+end_src

Here, we've specified that the tree-sitter compiler will be creating a file
named =libtree-sitter-javascript.so= (or =.dylib= or =.dll=), based on the =:lang=
field.  The corresponding tree-sitter mode in Emacs is called =js-ts-mode=, and
all of =js2-mode=, =js-mode=, and =javascript-mode= should attempt switching to the
=js-ts-mode=, if possible.

Moreover, since =js-2-mode= is first under the =:remap= section, that is the
"primary fallback."  Meaning that if the tree-sitter grammar is not available,
it will be the first mode tried.  If that doesn't work, it will try =js-mode=, and
=javascript-mode=, in that order, until one /does/ work.  If only one fallback needs
to be specified, a single quoted symbol is also acceptable.  For instance,
=python-ts-mode= just uses =:remap 'python-mode= in this argument position.

If a grammar mandates any other grammars be installed as a dependency, the
=:requires= keyword can specify a language symbol or list of symbols that should
be installed.  One example of this is found in the TypeScript recipe, which
specifies =:requires 'tsx=, since activating =typescript-ts-mode= on some Emacs
builds will attempt to load the TSX grammar.

The =:url=, =:revision=, =:source-dir=, =:cc=, and =:c++= arguments are all documented
under =treesit-language-source-alist=, which is part of base Emacs, not this
package.

** Keep track of your hooks
This package does not modify any of your major mode hooks.  That is, if you have
functions in =python-mode-hook=, but not in =python-ts-mode-hook=, then your hook
from =python-mode= will not be applied, assuming =python-ts-mode= is what gets
loaded.  For major modes in which this is a concern, the current recommendation
is to address this as part of your configuration.

#+begin_src emacs-lisp
(setq rust-ts-mode-hook rust-mode-hook)
#+end_src

Some modes have a shared base, such as =python-ts-mode= and =python-mode= both
deriving from =python-base-mode=.  For these languages, you can opt to hook into
=python-base-mode-hook= instead of explicitly setting the tree-sitter mode's hook.

** Automatically register extensions for =auto-mode-alist=
You can register tree-sitter modes to =auto-mode-alist= by calling
=treesit-auto-add-to-auto-mode-alist=.  Depending on the optional argument
=langs=, this function can behave in three different ways:

1. =nil=, the default - Only add tree-sitter modes to =auto-mode-alist= if the
   tree-sitter mode is available to Emacs /and/ the grammar is installed.
2. ='all= - For every tree-sitter mode available to Emacs and in
   =treesit-auto-langs=, add it to =auto-mode-alist= regardless of whether the
   grammar is installed.  This has the beneficial side effect of installing
   grammars for you when opening files that have a tree-sitter mode that comes
   with Emacs, but have no fallback mode.  Examples of this are =rust-ts-mode=,
   =go-ts-mode=, and a few others in Emacs 29+.
3. *A list of language symbols* - behaves like ='all=, but only for the listed
   languages

For instance, you might run this function as:

#+begin_src emacs-lisp
(treesit-auto-add-to-auto-mode-alist '(rust go toml))
#+end_src

This registers your tree-sitter modes according to the common file extension for
Rust, Go, and TOML, but no other modes.  Most users will probably want to use
=(treesit-auto-add-to-auto-mode-alist 'all)= for the easiest general behavior;
always prompting/installing grammars when we can.

* A rough vanilla equivalent
I find it a good practice to be skeptical of adding any new dependency to my
Emacs configuration.  So, what would you have to do to get similar behavior in
your Emacs configuration without the =treesit-auto= package?

We'll take two examples: TypeScript and Python.

Emacs 29 ships with =typescript-ts-mode= and =tsx-ts-mode=, but no equivalent
=typescript-mode= or =tsx-mode=.  =python-ts-mode= and =python-mode=, on the
other hand, are both available out of the box.  If you wanted these grammars to
automatically install on launch, and then use the tree-sitter modes instead of
the base modes for every file, you'd need the following code in your init file:

#+begin_src emacs-lisp
  (setq treesit-language-source-alist
        '((typescript . ("https://github.com/tree-sitter/tree-sitter-typescript" "master" "typescript/src"))
          (tsx . ("https://github.com/tree-sitter/tree-sitter-typescript" "master" "tsx/src"))
          (python . ("https://github.com/tree-sitter/tree-sitter-python"))))

  (dolist (source treesit-language-source-alist)
    (unless (treesit-ready-p (car source))
      (treesit-install-language-grammar (car source))))

  (add-to-list 'auto-mode-alist '("\\.ts\\'" . typescript-ts-mode))
  (add-to-list 'auto-mode-alist '("\\.tsx\\'" . tsx-ts-mode))
  (add-to-list 'major-mode-remap-alist '(python-mode . python-ts-mode))

#+end_src

There are plenty of reasons why some users would prefer to take this type of
hand-tuned approach to their tree-sitter modes.  For most Emacs people, though,
you can see the natural progression of where a config like the above would go:

+ You need to maintain a library of language symbols (which /must/ match the
  language's =grammar.js=), URLs, revisions, and sub-directories for
  =treesit-language-source-alist=.  Additionally, you'd need to know that
  =typescript-ts-mode= relies on the tree-sitter grammar for =tsx=, which must
  be installed alongside the grammar for TypeScript.  This is probably the main
  value proposition of this package, since all of this information is stored
  under the community-contributed =treesit-auto-recipe-list=
+ You need to be aware of when to use =auto-mode-alist= and
  =major-mode-remap-alist=, depending on what modes are installed into Emacs
+ If you want lazy installation of grammars, rather than installing them all
  up-front in your =init.el=, you'd need to write some hooks or derived modes to
  check =treesit-ready-p= for the current buffer, and install the language
  before loading the main tree-sitter mode
+ After a certain number of langauges, it becomes unweildy to do these
  =add-to-list= calls for every single one, and it's better to programmatically
  operate on a list of selected languages

If you were to follow this chain yourself, you'd probably wind up with something
in your =init.el= that looks very similar to the code in this package.

All in all, this is a small package.  Roughly half of it is just maintaining a
library of information for =treesit-language-source-alist=, and the other half
works through the logic of handling edge cases related to the remaining bullets
above.


* Caveats
This package is, admittedly, a hack.  =treesit.el= provides an excellent
foundation to incremental source code parsing for Emacs 29, and over time that
foundation /will/ expand into an improved core editing experience.  With that in
mind, I fully expect this package to eventually be obsolesced by the default
options in Emacs 30 and beyond.  That does not preclude us from adding a few
quality of life improvements to Emacs 29, though, and so it still seems prudent
to have this plugin available in the meantime.

* Known bugs
=treesit-auto= doesn't play super well with Org-babel, since Org has its own
methods of hooking into and using languages.  In particular, you may need to set
=org-src-lang-modes= yourself to get tree-sitter modes working correctly.

Another side behavior you may encounter is when opening an Org document with
shell scripts inside and =treesit-auto-install= is non-nil, then =treesit-auto=
will prompt to install the Bash grammar, but won't display the prompt until you
interact with Emacs in some way, such as using =next-line= (=C-n= by default) or
hovering over Emacs with your mouse.

* Contributing
Bug reports, feature requests, and contributions are most welcome.  Even though
this is a small project, there is always room for improvement.  I also
appreciate "nitpicky" contributions, such as formatting, conventions, variable
naming, code simplification, and improvements to language in documentation.

Issues are tracked on [[https://github.com/renzmann/treesit-auto/issues][GitHub]], which is also where patches and pull requests
should be submitted.

If you would like to submit a new language recipe to be distributed as part of
this package, see [[CONTRIBUTING.org][CONTRIBUTING.org]] for a quick guide on how to write and submit
the new recipe.
