Use CI to improve the quality of emacs distribution
In Better Emacs package development workflow, I proved that running Emacs compiler in CI can give huge boost to package quality.
The same workflow can apply to emacs distributions. But designing a CI workflow for the distribution is much more challenging.
For example, my .emacs.d uses about 300 packages. In CI pipeline, those packages are automatically downloaded and compiled. If compiling errors/warnings from those third party packages are not ignored, the CI will always fail.
There are also many other engineering issues. I struggled for five hours and finally got satisfying solution. Now anyone can use my solution to set up same CI pipeline in ten minutes.
Makefile is in "~/.emacs.d", it's still simple,
EMACS ?= emacs RM = @rm -rf EMACS_BATCH_OPTS = --batch -Q --debug-init install: clean @$(EMACS) $(EMACS_BATCH_OPTS) -l init.el compile: install @$(EMACS) $(EMACS_BATCH_OPTS) -l init.el -l tests/my-byte-compile.el 2>&1 | grep -Ev "init-(hydra|evil).el:.*Warning: docstring wider than 80 characters|an obsolete" | grep -E "[0-9]: ([Ee]rror|[Ww]arning):" && exit 1 || exit 0
You can run
make compile && echo good || echo bad in shell to test the pipeline locally. Please note I use
grep -v things-to-ignore to ignore some warnings. The warnings are from anonymous functions created by third packages (hydra, general.el, …).
The final missing piece is "~/.emacs.d/tests/my-byte-compile.el",
(require 'find-lisp) (require 'scroll-bar) (require 'ivy) (require 'counsel) (require 'w3m) (require 'ibuffer) (require 'org) (require 'diff-mode) (require 'cliphist) (require 'eacl) (require 'tramp) (require 'dired) (require 'shellcop) (require 'counsel-etags) (require 'typewriter-mode) (require 'pomodoro) (require 'emms) (require 'emms-playlist-mode) (require 'gnus) (require 'gnus-sum) (require 'gnus-msg) (require 'gnus-topic) (require 'magit) (require 'magit-refs) (require 'gnus-art) (require 'git-link) (require 'ace-window) (require 'js2-mode) (require 'yasnippet) (require 'ediff) (require 'company) (require 'evil-nerd-commenter) (require 'git-timemachine) (require 'pyim) (require 'cal-china-x) (require 'wucuo) (require 'langtool) (require 'web-mode) (require 'bbdb) (require 'gmail2bbdb) (require 'org-mime) (require 'pdf-tools) (require 'recentf) (require 'bookmark) (require 'find-file-in-project) (require 'flymake) (require 'elec-pair) (require 'elpy) (require 'rjsx-mode) (require 'simple-httpd) (require 'vc) (require 'sdcv) (require 'wgrep) (require 'mybigword) (require 'yaml-mode) (require 'octave) (require 'undo-fu) (require 'wc-mode) (require 'exec-path-from-shell) (require 'dictionary) (require 'company-ispell) (require 'company-ctags) (require 'lsp-mode) (let ((files (find-lisp-find-files-internal "." (lambda (file dir) (and (not (file-directory-p (expand-file-name file dir))) (string-match "\\.el$" file) (not (member file '(".dir-locals.el" "package-quickstart.el" "company-statistics-cache.el" "custom-set-variables.el" "early-init.el"))))) (lambda (dir parent) (member dir '("lisp")))))) (dolist (file files) ;; (message "file=%s" file) (byte-compile-file file))) ;;; my-byte-compile.el ends here
As you can see,
- I need add lots of
requirestatement to make compiling succeed on Emacs 26, 27, 28
- Some "*.el" files generated by Emacs and 3rd party packages need be ignored
- My emacs setup code is only in "lisp" directory
You can visit https://github.com/redguardtoo/emacs.d for a real world example.
BTW, you can find init-no-byte-compile.el where there are a few lines setup code the compiler will ignore. It's bad practice but sometimes there is no other way.
"init-no-byte-compile.el" is like,
;; -*- coding: utf-8; lexical-binding: t; -*- ;; blah blah ;; Local Variables: ;; no-byte-compile: t ;; End: