# Checker Framework developer manual

If you wish to use the Checker Framework, see its user manual (HTML, PDF).

This document contains information for Checker Framework developers, including people who wish to edit its source code or make pull requests.

Contents:

- Directory structure
  - Related repositories
- Build tasks
- Testing the Checker Framework
  - Testing optimizations
- Code style
- IDE configuration
- Pull requests
  - Branches
  - Pull request and commit notes for maintainers
- GitHub configuration
- Continuous Integration
  - Azure Pipelines
    - Increasing quota
    - How to change project visibility settings
  - Travis CI
  - Reproducing Continuous Integration build failures
- Documenting refactoring ideas
- Version numbers for annotated libraries
- Making a Checker Framework release
- Supporting a new version of Java
- Describing a case study
- Counting annotations
- Building a historical version of the Checker Framework

## Directory structure

The checker-framework repository contains several related projects:

`framework`
  the framework that enables building pluggable type checkers
`checker`
  the type checkers provided with the Checker Framework
`javacutil`
  utilities for integrating with javac
`dataflow`
  a dataflow framework that is used by the Checker Framework, Error Prone, NullAway, and other tools

The repository also contains the following directories:

`docs`
  documentation: manual, tutorial, examples, developer docs

### Related repositories

The `checker-framework` project depends on the `typetools/jdk` project, the `annotation-tools` project, and the `stubparser` project. When making changes to one of these projects, you may also need to make changes to one or more of the others.

If you make related changes in the `checker-framework` and `jdk` repositories, use the same branch name for each. The continuous integration framework will find and use that branch when running tests. For example, when continuous integration runs for branch B of fork F of `checker-framework`, it will use branch B of fork F of the other repositories (if they exist). The same is true for the other projects.

You do not need to explicitly build the `jdk` repository. Building the Checker Framework (say, via `./gradlew assemble`) automatically makes the Checker Framework use the annotated JDK that exists at `../jdk/` — that is, as a sibling directory of the Checker Framework named `jdk`.

If a change spans multiple projects, make pull requests for all of them. Each pull request description's should link to all the others.

Furthermore, whenever you make a pull request from a `checker-framework` branch A into branch B, if B has a corresponding `jdk` branch, then A also needs one. A needs the branch even if A's branch is identical to B's; in that case, beware of a bug in Azure Pipelines.

## Build tasks

Full instructions for building the Checker Framework from sources appear in the Checker Framework manual. This section describes the build system (the Gradle build tasks).

Don't run the `gradle` command, which would use whatever version of Gradle is installed on your computer. Instead, use the `gradlew` script in the `checker-framework` directory, also known as the Gradle wrapper.

Frequently-used tasks:

- `assemble`: builds all artifacts, including jars and Javadoc jars.
- `assembleForJavac`: builds only the jars required to use `checker/bin/javac`; faster than `assemble`.
- `build`: `assemble`, plus runs all JUnit tests.
- `allTests`: runs all tests.
- `spotlessApply`: reformats Java files.
- `spotlessCheck`: checks formatting of Java files.
- `NameOfJUnitTest`: runs the JUnit test with that name; for example, `NullnessTest`.
- `task`: lists tasks; use `--all` to see all tasks.

If you run a task from the main directory, then it will run that task in all subprojects with a task by that name. So, if you run `./gradlew allTests` that runs all tests everywhere. But `(cd framework && ../gradlew allTests)` only runs tests in the `framework` project. Alternatively, running `:framework:allTests` from the main directory or any subproject runs the `allTests` task only in the `framework` project.

An iterative build-test-debug cycle where you assemble the Checker Framework and execute its tests can sometimes lead to a state where Gradle is not able to compile and build the system for no obvious reason. This is especially common when you kill tests before they fully finish, or kill compilation before it is fully complete, leading to a bad state or an inconsistent build cache.

Below are some steps you might try to resolve this issue (in order):

- `./gradlew --no-build-cache assemble`: attempts to rebuild the Checker Framework without the build cache.
- Kill any Gradle daemons; run `./gradlew --status` to get a list of pids for Gradle.
- `git clean -fdx`: removes all files that are not under version control. Add or stash files that you would like to keep before running this command.
- Delete the Gradle build cache, usually located at `~/.gradle/caches`. Note: this will delete the entire build cache, including the cache for other Gradle projects on your machine; the build step for the Checker Framework will need to re-download all required dependencies.

## Testing the Checker Framework

For writing new test cases, see file `checker/tests/README`.

### Testing optimizations

To test an optimization that should speed up type-checking, see the `test-daikon.sh` stage of the `daikon_jdk*` job of the Azure Pipelines CI job. Compare the run time of this stage (or of the entire `daikon_jdk*` job) between the master branch and a branch with your improvements.

You can also compare run times of the Checker Framework test suite.

## Code style

Code in this project follows the Google Java Style Guide, Michael Ernst's coding style guidelines, and Oracle's Java code conventions.

From the command line, you can format your code by running `./gradlew spotlessApply`. You can configure your IDE (Eclipse or IntelliJ) to use the formatting.

We don't use `@author` Javadoc tags in code files. Doing so clutters the code, and is misleading once that individual is no longer maintaining the code. Authorship (and who has made changes recently) can be obtained from the version control system, such as by running `git annotate filename`.

Every class, method, and field (even private ones) must have a descriptive Javadoc comment.

If a nested class has a very common and generic name, do not `import` it. In code, write the outer and inner class name (as in `Tree.Kind` or `Map.Entry`) rather than just the simple name `Kind` or `Entry`.

## IDE configuration

First clone and build all projects from their sources, from the command line, using the instructions at https://checkerframework.org/manual/#build-source. After that succeeds, import the projects into your IDE as Gradle projects.

If your IDE cannot find `com.sun.source.*` packages, try changing the project JDK to JDK 11 or JDK 17.

For VS Code and Eclipse, run `./gradlew eclipseClasspath` before running the IDE for the first time. This ensures generated files are not put in the standard `../bin/` directories.

For VS Code and Eclipse, run `./gradlew eclipseClasspath` before running the IDE for the first time. This ensures generated files are not put in the standard `../bin/` directories.

## Pull requests

Each pull request should address a single concern, rather than (say) addressing multiple concerns such as fixing a bug, adding a feature, and changing formatting. Focusing each pull request on a single concern makes the commit easier to understand and review. It also makes the commit history (after the pull request is merged) easier to understand and (if necessary) revert.

The pull request title should clearly explain the change. It will be used as the commit message for your change. If the pull request fixes an issue in the issue tracker, its title should end with "; fixes #NNN" where NNN is the fixed issue.

Your pull request (whether it is a bug fix or a new feature) should include tests that fail before your change and pass afterward.

If you make a user-visible change, update the manual (or add a new section) and add a brief description at the top of the changelog (file `docs/CHANGELOG.md`).

To reduce iterations of code review, please follow the coding conventions. Also enable continuous integration on your fork of the Checker Framework and ensure that it passes before you open a pull request.

A pull request marked as "draft" means it should not be reviewed. To use a "draft" pull request for discussions, make it in a fork. If it were in the `typetools` GitHub organization, it would use CI resources and thus slow down CI feedback for other pull requests.

Also see Michael Ernst's advice about creating GitHub pull requests.

### Branches

It is good style to create a branch (in your fork of the Checker Framework GitHub repository) for each independent change. Do not make changes to your `master` branch. If you have write access to the `typetools` repository, don't work in a branch of it, because such a branch competes for CI resources with all pull requests.

You may need to make changes to multiple repositories. See "Related repositories" above.

Azure Pipelines has a bug: whenever two CI jobs would run code with the same commit hash, it re-uses a previous execution result. This is a bug because the CI job's behavior may depend on the branch name and other factors that are independent of the commit hash. This means that you may see spurious successes or failures when your branch of (say) the `jdk` repository has the identical commit hash to some other branch that Azure Pipelines previous ran a job for.

### Pull request and commit notes for maintainers

It is acceptable to commit small, noncontroversial changes directly to master. (This policy differs from some projects, which require an issue tracker issue and a pull request for every change, however minor.) As with pull requests, each commit should address a single concern. For any change where you want feedback, or where others might have useful comments or might disagree, please submit a pull request. Be conservative in your judgment; others might consider something controversial that you do not.

Try to review pull requests promptly, to avoid stalling others while waiting for your feedback. If you have been waiting for more than a week after the pull request was assigned with no feedback, then ping the assignee, wait at least another business day, and then go ahead and push your changes. It's great if reviewers can give feedback, but if they are too busy to do so, you should recognize that and move on.

## GitHub configuration

When you installed Git, you should have set your name and email address. If you have not yet done so, do it now:

    git config --global user.name "FIRST_NAME LAST_NAME"
    git config --global user.email "USERNAME@SOMEDOMAIN.COM"

Before you make any commits (even to your own fork), update your GitHub account profile so it contains your complete name. This is necessary to include you in the list of contributors.

You will want your own GitHub fork of any project you plan to modify. We recommend that, for each fork, you configure GitHub to delete branches after they are merged.

## Continuous Integration

The Checker Framework has continuous integration jobs that run in Azure Pipelines, CircleCI, and/or Travis CI on each push to GitHub. We recommend Azure Pipelines.

### Azure Pipelines

To enable Azure Pipelines continuous integration for your fork: (This is a summary of the Azure Pipelines getting started directions.)

- Browse to dev.azure.com. (You might need to create a (free) account.)
- Click "Create a project to get started" and enter an appropriate name
- Click "public".
  - If this option is not available, create the project and modify its visibility later.
- (You might have to wait a few days for your project to be created.)
- Click "create project"
- At the left, click the blue rocket labeled "pipelines"
- Click "new pipeline"
- Click "GitHub" (You might need to authorize the app on GitHub.)
- Click "Select a repository"
- Choose MYUSERID/checker-framework
- Choose the default radio button, "only selected repositories". Do not choose "all repositories".
- Choose "Existing Azure Pipelines YAML file"
- Select "/azure-pipelines.yml" in the dropdown menu
- Approve and install.
- Click "run".

#### Increasing quota

By default, your project has no parallelism and thus pipeline runs fail immediately. Azure DevOps provides a Microsoft Forms through which you can request increased public project concurrency for validated open-source projects.

It may be the case that you obtain only 1800 minutes per month, only one parallel job at a time, and only for private projects. In this case, you need to ask Azure to lift all three restrictions. Send mail such as the following:

Thank you for granting me free tier access to Azure Pipelines. I am working on the open-source Checker Framework. Therefore could you please make the following changes, per the policies at https://azure.microsoft.com/en-us/products/devops/pipelines/?

- Increase parallelism beyond 1 job.
- Increase time limit beyond 1800 minutes per month.
- Permit the project to be public, for collaboration.

Thanks for your help.

#### How to change project visibility settings

To change your project's visibility settings (usually to "Public" to enable CI):

- Browse to your project on dev.azure.com.
- Click the "Project Settings" button.
- Click the "Visibility" dropdown.
- Click "Public".

### Travis CI

To enable Travis CI continuous integration for your fork:

- Browse to travis-ci.com
- Click the downarrow for the dropdown, near your face in the upper right corner
- Click settings
- Find "checker-framework" in the list, and click the slider to enable it.

### Reproducing Continuous Integration build failures

If a CI job fails, examine the CI logs.

You can also run the same test locally. Each CI job runs a different command. You can see the commands in file `azure-pipelines.yml`, usually on lines starting with `bash:`. The scripts that are run generally just invoke gradle to run particular tasks.

Sometimes, CI tests for your pull request may fail even though the same command passed locally. First, ensure you are using the same JDK. Second, all CI tests are run in a Docker container, and you can use that container if necessary. The container names are in top-level file `azure-pipelines.yml`, on lines starting with `container:`.

## Documenting refactoring ideas

Don't open issues for code improvement ideas (such as potential refactorings). If it can be described concisely and is unlikely to be rediscovered by other people, write a TODO comment in the code. The code comment is more likely to be noticed by someone working with the code, and it is equally easy to search for. Furthermore, it doesn't clutter the issue tracker. Clutter in the issue tracker reduces morale, makes it harder to search, and makes the project appear lower-quality than it actually is.

## Version numbers for annotated libraries

We maintain annotated versions of some third-party libraries. The source code appears in a fork in the GitHub `typetools` organization. Binaries are hosted at Maven Central in the `org.checkerframework.annotatedlib` group.

Annotated libraries should be based on a released version of the upstream library, not an arbitrary commit in the upstream library's version control system. The library's version number is the same as the upstream version number.

When making a new version of an annotated library, between upstream releases, add ".0.1" to the end of the version number. For example, if we already uploaded version 6.2 to Maven Central, the next version we upload would be 6.2.0.1. This accommodates the possibility that the upstream maintainers release 6.2.1. Our further releases increment the last number, for example to 6.2.0.2.

## Making a Checker Framework release

See a separate document about the Checker Framework release process: web version (from previous release) or local version (link works from a clone).

## Supporting a new version of Java

To upgrade the Checker Framework to use a newer version of Java:

    # Create a branch named "java-23".
    cd $t/checker-framework-fork-mernst-branch-master
    gnb java-23
    cd $t/checker-framework-fork-mernst-branch-java-23

    # Edit:
    checker/bin/wpi.sh
    checker/bin/wpi-many.sh (diff it against wpi.sh)
    # Rename and edit:
    checker/bin-devel/Dockerfile-ubuntu-jdkXX
    checker/bin-devel/Dockerfile-ubuntu-jdkXX-plus (diff it against Dockerfile-ubuntu-jdkXX)
    # Further search for: (java|jdk).?22\b
    # Don't push changes yet.

    # Use the latest version of Gradle, which supports the new JDK version
    # (see https://docs.gradle.org/current/userguide/compatibility.html):
    for wmtest in wpi-many-tests-bcel-util wpi-many-tests-bibtex-clean wpi-many-tests-ensures-called-methods wpi-many-tests-html-pretty-print wpi-many-tests-owning-field -wpi-many-tests-bibtex-clean ; do
      cd $t/$wmtest && \
      (./gradlew wrapper --gradle-version 8.10 && ./gradlew build --warning-mode=all) && \
      git commit -m "Use Gradle 8.10" gradle/wrapper/gradle-wrapper.properties && \
      git push
    done
    # Now update file checker/tests/wpi-many/testin.txt.

    # Build the Docker images for the newest JDK version (others need not be rebuilt).

    # Push changes.  Fix any build failures.

## Describing a case study

After you have performed a case study of applying some checker to an open-source codebase, you should have:

- the unannotated code, as of the time/commit you started annotating it. This might be in your forked repository, or in an upstream repository.
- the annotated code. Typically, this is in a branch of your forked repository.

  In the annotated code, each `@SuppressWarnings` annotation should have a brief justification, explaining why the code is correct and the warning is a false positive. The justification should be a `//`-style comment, on the same line as the argument to `@SuppressWarnings`. For example:

        @SuppressWarnings("nullness:assignment") // dynamic check: checked against null immediately above

        @SuppressWarnings({
          "nullness:assignment" // dynamic check: checked against null immediately above
        })

  If there are more than about 10 warning suppressions, prefix each one by a category followed by a colon (as with "dynamic check:") above, to aid in computing statistics about the causes of false positive warnings.
- the command to run the type-checker(s) on the annotated code.
- a list of all the bugs you fixed, with a brief description of each. The description can point to pull requests, or commits, or just be text; provide whatever is most helpful.

## Counting annotations

After you have annotated a project, you may wish to count the annotations that you have written. These programs will help you do that:

- AnnotationStatistics
- JavaCodeStatistics
- ReportChecker
- count-suppression-reasons

## Building a historical version of the Checker Framework

The use of four different repositories, as explained in "Related repositories" above, means that you cannot just check out an old version of the Checker Framework and build it with `./gradlew assemble`. By default, the Checker Framework build uses the latest version of the other three repositories, which are rarely compatible with an old version of the Checker Framework. One symptom of the problem is a "Can't find stubparser jar" error message.

To build an old version of the Checker Framework, you need to use old versions of the other repositories as well. Clone them into siblings of the `checker-framework/` directory that holds your old version of the Checker Framework.

Here are general rules: If the Checker Framework version is a commit on branch B with date D, then use the commits on other repositories' branch B (or `master`, if branch B does not exist) that precede D. (This isn't quite right, because the commits may have been made earlier but all pushed at the same time, so you might need to look at commits that are relatively soon after D.) To obtain these, view the GitHub history and click on the pull request to see the commits that were merged into it. You cannot obtain these from the Git history, because after a pull request is merged, the commits in the pull request branch are squashed and the branch is deleted.

Here are special cases:

- If you are building an old released version of the Checker Framework, then `annotation-tools` should have a corresponding tag for the release.
- Recently, any Stubparser change that breaks building the Checker Framework, should have a new version number. So you should check out Stubparser to the commit that changed the version number (to what it is in `checker-framework/build.gradle`).

The script `checker/bin-devel/checkout-historical.sh` approximates these rules. It is able to build the Checker Framework back through at least mid-April 2019. It cannot build the Checker Framework as of mid-January 2019.