Use Magit to commit efficiently and correctly

I prefer using git cli because it's more light weight.

Here is my bash alias of git commit,

alias gc="git commit -m"

The problem of my "cli-only" workflow is it can't detect my mistakes automatically.

I often forget to add new code file into git. So my final commit might miss files.

Magit UI solution

One solution is to use Magit to commit inside Emacs. After commit, I could double check the files inside the hooks provided by Magit.

My set up in Emacs,

(defun my-lines-from-command-output (command)
  "Return lines of COMMAND output."
  (let* ((output (string-trim (shell-command-to-string command)))
         (cands (nonempty-lines output)))
    (delq nil (delete-dups cands))))

(defun my-hint-untracked-files ()
  "If untracked files and commited files share same extension, warn users."
  (let* ((exts (mapcar 'file-name-extension (my-lines-from-command-output "git diff-tree --no-commit-id --name-only -r HEAD")))
         (untracked-files (my-lines-from-command-output "git --no-pager ls-files --others --exclude-standard"))
         (lookup-ext (make-hash-table :test #'equal))
         rlt)
    ;; file extensions of files in HEAD commit
    (dolist (ext exts)
      (puthash ext t lookup-ext))
    ;; If untracked file has same file extension as committed files
    ;; maybe they should be staged too?
    (dolist (file untracked-files)
      (when (gethash (file-name-extension file) lookup-ext)
        (push (file-name-nondirectory file) rlt)))
    (when rlt
      (message "Stage files? %s" (mapconcat 'identity rlt " ")))))

(with-eval-after-load 'magit
  (defun my-git-check-status ()
    "Check git repo status."
    ;; use timer here to wait magit cool down
    (run-with-idle-timer 1 nil #'my-hint-untracked-files))
  (add-hook 'magit-post-commit-hook #'my-git-check-status)
  (add-hook 'git-commit-post-finish-hook #'my-git-check-status))

Screenshot of step 1 in Emacs, magit-commit-step1.png

Screenshot of step 2 (final step) in Emacs (I was reminded of untracked files "bye.js" and "tree.js" at the bottom of UI), magit-commit-step2.png

BTW, my actual code in my .emacs.d is a bit different.

CLI solution but still using Magit

Another solution is still doing the git thing in shell but use the Magit plus Emacs "-batch" option.

Here is my bash setup,

function gc {
    # check my emacs.d exist
    if [ -f "$HOME/.emacs.d/README.org" ] && [ "$PWD" != "$HOME/.emacs.d" ]; then
        # magit hook does not work
        emacs -batch -Q -l "$HOME/.emacs.d/init.el" --eval "(magit-commit-create '(\"-m\" \"$@\"))" --eval "(my-hint-untracked-files)"
    else
        git commit -m "$@"
    fi
}

Screenshot in shell, magit-commit-in-shell.png

Linux audio input configuration

Run sudo alsamixer and press "F4" to choose audio input.

Make sure the right "Rear Mic" and "Capture" are enabled, linux-audio-input.png

Run alsamixer and check "pulseaudio" configuration in the same way.

Then test audio,

arecord --duration=5 --format=dat test-mic.wav && aplay test-mic.wav

Hardcore spell checking in Emacs

This article is not introduction of Emacs spell checking basics. It requires deep knowledge of Emacs Lisp and Fly Spell.

You could read my article What's the best spell check setup in emacs for basic knowledge.

This article introduces new techniques to make Fly Spell more powerful and faster.

The CLI programs aspell and hunspell can only parse plain text. They don't know any programming language syntax.

Fly Spell feeds the output of CLI program into its own Lisp predicate named flyspell-generic-check-word-predicate whose default value is nil.

When executing (flyspell-mode 1), the per mode predicate is assigned to flyspell-generic-check-word-predicate.

For example, you can run (get major-mode 'flyspell-mode-predicate) to get predicate of current major mode, (get 'web-mode 'flyspell-mode-predicate) to get predicate of web-mode.

The predicate is a simple function without parameter. Here is my predicate for web-mode,

(defun my-web-mode-flyspell-verify ()
  "Fly Spell predicate of `web-mode`."
  (let* ((font-face-at-point (get-text-property (- (point) 1) 'face))
         rlt)
    ;; If rlt is t, the word at point is POSSIBLY a typo, continue checking.
    (setq rlt t)
    ;; if rlt is nil, the word at point is definitely NOT a typo.
    ;; (setq rlt nil)
    rlt))
;; Attach my predicate to `web-mode`
(put 'web-mode 'flyspell-mode-predicate 'my-web-mode-flyspell-verify)

If you read code of flyspell-prog-mode, you will find it set flyspell-generic-check-word-predicate to its own predicate flyspell-generic-progmode-verify,

(defvar flyspell-prog-text-faces
  '(font-lock-string-face font-lock-comment-face font-lock-doc-face)
  "Faces corresponding to text in programming-mode buffers.")

(defun flyspell-generic-progmode-verify ()
  "Used for `flyspell-generic-check-word-predicate' in programming modes."
  (unless (eql (point) (point-min))
    ;; (point) is next char after the word. Must check one char before.
    (let ((f (get-text-property (1- (point)) 'face)))
      (memq f flyspell-prog-text-faces))))

As you can see, flyspell-generic-progmode-verify is very simple. If the word at point is not inside comment or string, the predicate returns nil which means the word is not a typo.

So in theory I can write my own predicate by following flyspell-generic-progmode-verify.

But in reality it's not as simple as it seems. The predicate is written in Lisp so it's slow. If it contains too much code, Fly Spell process might block other actions in Emacs. Emacs could be un-responsive when editing text.

The solution is not to start Fly Spell process too frequently.

The flyspell-mode starts checking when text in current buffer is modified.

My solution is not to turn on flyspell-mode. Instead, I manage the spell checking by myself using APIs from flyspell.

I only spell check when user saving current buffer. The interval between spell check should not be less than 5 minutes. Spell check is done by calling API flyspell-buffer

Checking the whole buffer is still slow. Instead, we can check the text region in current window by calling flyspell-region instead. The api window-total-height returns the height of current Windows. So I can use below code to get the region to check,

(let* (beg end (orig-pos (point)))
  (save-excursion
    (forward-line (- (window-total-height)))
    (setq beg (line-beginning-position))
    (goto-char orig-pos)
    (forward-line (window-total-height))
    (setq end (line-end-position)))
  (flyspell-region beg end))

I also need respect the predicate embedded in the major mode in my own generic predicate. Since per mode predicate has already checked the font face, I should skip the font face check in generic predicate if per mode predicate exists.

Above algorithms are implemented in wucuo.

Usage,

(add-hook 'prog-mode-hook 'wucuo-start)
(add-hook 'text-mode-hook 'wucuo-start)

If wucuo-flyspell-start-mode is "fast" (default value), flyspell-region is used, visible region is checked when user saves current file.

If wucuo-flyspell-start-mode is "normal", flyspell-buffer is used, current buffer is checked when user saves current file.

Audio recording on Linux

  • Run sudo alsamixer and turn off mic to reduce the noise
  • Run alsamixer to double check pulse setup
  • Make sure correct device is selected in audacity
  • Restart audacity and test

My alsamixer setup, alsamixer-nq8.png

Use Magit API to rebase to closest branch

My workflow in Git is,

  • Create a new feature branch based on main branch
  • Add some small commits into feature branch
  • Rebase feature branch interactively

The final rebase step happens a lot.

So I could use Magit api magit-rebase-interactive to speed up it.

The key is to analyze output of git log --decorate --oneline to find the main branch commit.

Code,

(defun my-git-extract-based (target)
  "Extract based version from TARGET."
  (replace-regexp-in-string "^tag: +"
                            ""
                            (car (nreverse (split-string target ", +")))))

(defun my-git-rebase-interactive (&optional user-select-branch)
  "Rebase interactively on the closest branch or tag in git log output.
If USER-SELECT-BRANCH is not nil, rebase on the tag or branch selected by user."
  (interactive "P")
  (let* ((log-output (shell-command-to-string "git --no-pager log --decorate --oneline -n 1024"))
         (lines (split-string log-output "\n"))
         (targets (delq nil
                        (mapcar (lambda (e)
                                  (when (and (string-match "^[a-z0-9]+ (\\([^()]+\\)) " e)
                                             (not (string-match "^[a-z0-9]+ (HEAD " e)))
                                    (match-string 1 e))) lines)))
         based)
    (cond
     ((or (not targets) (eq (length targets) 0))
      (message "No tag or branch is found to base on."))
     ((or (not user-select-branch)) (eq (length targets) 1)
      ;; select the closest/only tag or branch
      (setq based (my-git-extract-based (nth 0 targets))))
     (t
      ;; select the one tag or branch
      (setq based (my-git-extract-based (completing-read "Select based: " targets)))))

    ;; start git rebase
    (when based
      (magit-rebase-interactive based nil))))

Screencast:

magit-rebase-api.gif

Make Emacs faster than Vim in "git mergetool"

My article Emacs is the best merge tool for Git explains how to combine git mergetool with ediff-mode in Emacs.

Harrison McCullough suggested the work flow can be faster if emacs is replaced with emacsclient.

I did some research and found a perfect solution. It's even faster than Vim.

Initial solution

Please note emacsclient is only use for resolving conflicts.

Step 1, start emacs server by running emacs -Q --daemon --eval "(setq startup-now t)" -l "/home/my-username/.emacs.d/init.el" --eval "(progn (require 'server) (server-start))" in shell.

Step 2, insert below code into ~/.emacs.d/init.el (see the comment why this advice is required):

(defadvice server-save-buffers-kill-terminal (after server-save-buffers-kill-terminal-after-hack activate)
  ;; kill all buffers, so new ediff panel is re-created and `ediff-startup-hook-setup' is called again
  ;; besides, remove the buffers whose binding files are alredy merged in `buffer-list'
  (mapc 'kill-buffer (buffer-list)))

Step 3, insert below code into ~/.gitconfig:

[mergetool.ediff]
cmd = emacsclient -nw --eval \"(progn (setq ediff-quit-hook 'kill-emacs) (if (file-readable-p \\\"$BASE\\\") (ediff-merge-files-with-ancestor \\\"$LOCAL\\\" \\\"$REMOTE\\\" \\\"$BASE\\\" nil \\\"$MERGED\\\") (ediff-merge-files \\\"$LOCAL\\\" \\\"$REMOTE\\\" nil \\\"$MERGED\\\")))\"

My real world solution

It's similar to initial solution. But some scripts are created for automation.

Step 1, read Using Emacs as a Server in the manual and create ~/.config/systemd/user/emacs.service for Systemd:

[Unit]
Description=Emacs text editor
Documentation=info:emacs man:emacs(1) https://gnu.org/software/emacs/

[Service]
Type=forking
ExecStart=emacs -Q --daemon --eval "(setq startup-now t)" -l "/home/my-username/.emacs.d/init.el" --eval "(progn (require 'server) (server-start))" 
ExecStop=emacsclient --eval "(kill-emacs)"
Environment=SSH_AUTH_SOCK=%t/keyring/ssh
Restart=on-failure

[Install]
WantedBy=default.target

Step 2, set up in ~/.gitconfig:

[mergetool.emacs]
    cmd = ediff.sh "$LOCAL" "$REMOTE" "$BASE" "$MERGED"
[mergetool.emacsclient]
    cmd = MYEMACSCLIENT=emacsclient ediff.sh "$LOCAL" "$REMOTE" "$BASE" "$MERGED"

Step 3, create ediff.sh:

#!/bin/sh
[ -z "$MYEMACSCLIENT" ] && MYEMACSCLIENT="emacs"
# emacsclient won't work in git mergetool
# $1=$LOCAL $2=$REMOTE $3=$BASE $4=$MERGED
if [ "$MYEMACSCLIENT" = "emacs" ]; then
    $MYEMACSCLIENT -nw -Q --eval "(setq startup-now t)" -l "$HOME/.emacs.d/init.el" --eval "(progn (setq ediff-quit-hook 'kill-emacs) (if (file-readable-p \"$3\") (ediff-merge-files-with-ancestor \"$1\" \"$2\" \"$3\" nil \"$4\") (ediff-merge-files \"$1\" \"$2\" nil \"$4\")))"
else
    $MYEMACSCLIENT -nw --eval "(progn (setq ediff-quit-hook 'kill-emacs) (if (file-readable-p \"$3\") (ediff-merge-files-with-ancestor \"$1\" \"$2\" \"$3\" nil \"$4\") (ediff-merge-files \"$1\" \"$2\" nil \"$4\")))"
fi

Step 4, run git mergetool -t emacsclient to resolve conflicts.

My init-ediff.el in emacs.d.

Thoughts on "Native shell completion in Emacs"

Native shell completion in Emacs by Troy Hinckley is must read for completion in shell-mode.

One problem is my ~/.bashrc executes /etc/bash_completion,

if [ -f /etc/bash_completion ]; then
    . /etc/bash_completion
fi

Unfortunately /etc/bash_completion makes complete -p output some lines the Emacs function bash-completion-tokenize can't analyze.

Here is output of complete -p at my PC,

...
complete -F _known_hosts mtr
complete -o default -o nospace -W 'homepc
192.168.1.104
github.com
gitlab.com' scp
complete -o default -f -X '!*.dvi' dvipdf
...

The line gitlab.com' scp will crash bash-completion-tokenize. Obviously, one line complete -o default -o nospace -W 'homepc 192.168.1.104 github.com gitlab.com' scp is wrongly split into multiple lines by complete -p.

In shell-mode, completion functions might call bash-completion-tokenize. If bash-completion-tokenize crashes, the completion in shell-mode won't work.

Besides, if company-mode provides auto-completion UI, it's better to place the backend company-files before company-native-complete. It's because the backend company-files displays the full file path in candidates. So users can complete the whole path in one shot.

My setup code for the packages Troy Hinckley suggested,

;; Enable auto-completion in `shell'.
(with-eval-after-load 'shell
  (native-complete-setup-bash))

;; `bash-completion-tokenize' can handle garbage output of "complete -p"
(defadvice bash-completion-tokenize (around bash-completion-tokenize-hack activate)
  (let* ((args (ad-get-args 0))
         (beg (nth 0 args))
         (end (nth 1 args)))
    ;; original code extracts tokens from output of "complete -p" line by line
    (cond
     ((not (string-match-p "^complete " (buffer-substring beg end)))
      ;; filter out some wierd lines
      (setq ad-return-value nil))
     (t
      ad-do-it))))

(defun shell-mode-hook-setup ()
  "Set up `shell-mode'."
  ;; hook `completion-at-point', optional
  (add-hook 'completion-at-point-functions #'native-complete-at-point nil t)
  (setq-local company-backends '((company-files company-native-complete)))
  ;; `company-native-complete' is better than `completion-at-point'
  (local-set-key (kbd "TAB") 'company-complete))
(add-hook 'shell-mode-hook 'shell-mode-hook-setup)

Screenshot,

shell-complete-path-nq8.png

shell-complete-param-nq8.png

How to speed up lsp-mode

Here is my setup,

(with-eval-after-load 'lsp-mode
  ;; enable log only for debug
  (setq lsp-log-io nil)

  ;; use `evil-matchit' instead
  (setq lsp-enable-folding nil)

  ;; no real time syntax check
  (setq lsp-diagnostic-package :none)

  ;; handle yasnippet by myself
  (setq lsp-enable-snippet nil)

  ;; use `company-ctags' only.
  ;; Please note `company-lsp' is automatically enabled if installed
  (setq lsp-enable-completion-at-point nil)

  ;; turn off for better performance
  (setq lsp-enable-symbol-highlighting nil)

  ;; use ffip instead
  (setq lsp-enable-links nil)

  ;; auto restart lsp
  (setq lsp-restart 'auto-restart)

  ;; @see https://github.com/emacs-lsp/lsp-mode/pull/1498 and code related to auto configure.
  ;; Require clients could be slow.
  ;; I only load `lsp-clients' because it includes the js client which I'm interested
  (setq lsp-client-packages '(lsp-clients))

  ;; don't scan 3rd party javascript libraries
  (push "[/\\\\][^/\\\\]*\\.\\(json\\|html\\|jade\\)$" lsp-file-watch-ignored) ; json

  ;; don't ping LSP lanaguage server too frequently
  (defvar lsp-on-touch-time 0)
  (defadvice lsp-on-change (around lsp-on-change-hack activate)
    ;; don't run `lsp-on-change' too frequently
    (when (> (- (float-time (current-time))
                lsp-on-touch-time) 30) ;; 30 seconds
      (setq lsp-on-touch-time (float-time (current-time)))
      ad-do-it)))

(defun my-connect-lsp (&optional no-reconnect)
  "Connect lsp server.  If NO-RECONNECT is t, don't shutdown existing lsp connection."
  (interactive "P")
  (when (and (not no-reconnect)
             (fboundp 'lsp-disconnect))
    (lsp-disconnect))
  (when (and buffer-file-name
             (not (member (file-name-extension buffer-file-name)
                          '("json"))))
    (unless (and (boundp 'lsp-mode) lsp-mode)
      (if (derived-mode-p 'js2-mode) (setq-local lsp-enable-imenu nil))
      (lsp-deferred))))

To enable lsp for the major mode XXX-mode needs only one line,

(add-hook 'XXX-mode-hook #'my-connect-lsp)

You also need install three packages,

Explanation,

Ctags is used to generate tags file for company-ctags and counsel-etags. GNU Find is required for find-file-in-project.

These three packages are faster and can replace the corresponding functionalities in lsp-mode.

I don't need any lint tools from lsp-mode because the lint tool is already included in our build script. I can see the syntax error from terminal.

I advice the lsp-on-change in order to notify the language server less frequently.

js2-mode has its own javascript parser extract imenu items. So I don't need javascript language server's parser to send back imenu items.

By default lsp-client-packages contains many clients, but I only code in javascript which is included in lsp-clients.

Here is code quoted from lsp-mode,

;;;###autoload
(defun lsp (&optional arg)
  ;; ...
  (when (and lsp-auto-configure)
    (seq-do (lambda (package) (require package nil t))
            lsp-client-packages))
  ;; ...
)

I have done some profiling by insert (profiler-report-cpu) at the end of lsp (the bottlenecks is highlighted).

lsp-mode-bottleneck-nq8.png

The language server I used can read jsconfig.json in project root. I can specify the directories to exclude in it.

Yin and Yang in Emacs

As a Chinese, I studied Tao Te Ching since childhood. So I believe Tao (the way) exist in Emacs. Tao is basically Yin and Yang who lives in harmony in Emacs.

I can't say Yin is good and Yang is evil, or vice versa. All I can do is to find the way to make Yin and Yang co-exist.

For example, a few days ago I published Effective "git blame" in Emacs which introduced my package vc-msg.

It became one of my most popular reddit post because its unique feature partial line blame,

vc-msg-good.png

vc-msg-bad.png

I noticed some comments compared my package with Magit. Those comments were very educational and I did learn a few useful tricks.

My point is, vc-msg and Magit could collaborate without any problem, like Yin and Yang lives harmony. If you find any conflict between vc-msg and Magit, just let me know. I will fix it.

I totally understand there are many Magit lovers in Emacs community. So I make vs-msg v1.0.2 to support Magit. You can use partial line blame in vc-msg but calling Magit command to open the commit.

It's only one line setup,

(setq vc-msg-git-show-commit-function 'magit-show-commit)

vc-msg-and-magit.png

I tested in magit-blame-mode and found no issue.

I'm sure vc-msg should work in other major modes or minor modes. There are also two callback functions vc-msg-get-current-file-function and vc-msg-get-line-num-function which users can customize.

Effective "git blame" in Emacs

I published Emacs package vc-msg. It uses git-blame to show commit information of current line.

In the new version, it can display the correct commit information of current line.

For example, the line 6 at https://github.com/redguardtoo/test-git-blame/blob/master/hello.js is changed by three commits.

Select the partial of line 6 and run vc-msg-show, the correct commit is displayed.

Screenshots,

vc-msg-neutral.png

vc-msg-good.png

vc-msg-bad.png