Migrate blog from wordpress into nikola

UPDATED: <2015-01-31 Sat>

CREATED: <2014-04-22>

Here are the steps. Tested on Cygwin/Mac/Linux.

1 Requirements


# install python2 and make sure sqlite is supported
sudo USE="sqlite" emerge -a =python-2* # Gentoo Linux

Make sure gcc libxslt-devel libxml2-devel zlib-devel installed, especially on cygwin. See http://getnikola.com/handbook.html#libxml-libxslt-files-missing-errors.

2 Install

2.1 Install Nikola

Please note the package "requests" is required by zen theme, so it's optional.

I prefer install packages into my HOME directory because it versatile and safe.

For example, "sudo pip install" could screw up python on Gentoo Linux.


# @see http://stackoverflow.com/questions/2915471/install-a-python-package-into-a-different-directory-using-pip
pip install --user markdown webassets nikola requests
export PATH=$PATH:$HOME/.local/bin

2.2 Install theme (OPTIONAL)

I use zen theme,


rm -rf themes/zen/;nikola install_theme zen
# or rm -rf themes/zen/; http_proxy=http://127.0.0.1:8087 nikola install_theme zen at mainland China

Backup you theme under "~/.config/nikola/themes/zen". You won't need above command any more.

3 Usage

3.1 Create root directory


mkdir -p ~/.config/;nikola init "nikola";cd ~/.config/nikola;

Please note `nikola init "nikola"` creates "conf.py" for you. Backup this file and you never need `nikola init`.

3.2 Import old posts and build new site


nicola import_wordpress my_wordpress_dump.xml
nikola build

The website is create at "~/.config/nikola/output".

4 Tips (OPTIONAL)

4.1 If wordpress use 3rd parth JS to render code

Use below command to fix embedded code in HTML files:


find -name '*.wp' -exec grep -l "\[sourcecode.*\<diff\>.*\]" {} \; |xargs sed -i 's/\[sourcecode.*\<diff\>.*\]/<pre class="brush: diff;">/g
find -name '*.wp' -exec grep -l "~~~~~~~~~~~~" {} \;|xargs sed -i "s%~~~~~~~~~~~~%</pre>%g"

Manually fixed those articles with Chinese titles in url_map.csv

4.2 Import comments into Disqus

Disqus is a blog comment hosting service.

Use below script to fixed the comment dumped from wordpress before importing it into Disqus:


#!/usr/bin/python
import getopt, sys, csv
def usage():
    print '''
NAME
    fix url mapping when migrate wordpress blog into nikola
Usage
    python fix-url-map.py [options]
'''[1:-1]

if __name__ == '__main__':
    try:
        opts, args = getopt.getopt(sys.argv[1:], "hf:x:", ["help", "file=","xml="])
    except getopt.GetoptError as err:
        # print help information and exit:
        print str(err) # will print something like "option -a not recognized"
        usage()
        sys.exit(2)

    file=""
    xml=""

    for o, a in opts:
        if o in ("-h", "--help"):
            usage()
            sys.exit()
        elif o in ("-f", "--file"):
            file= a
        elif o in ("-x", "--xml"):
            xml=a
        else:
            assert False, "unhandled option"

    with open(xml, 'r') as content_file:
        content = content_file.read()

    with open(file, 'rb') as csvfile:
         spamreader = csv.reader(csvfile, delimiter=',')
         for row in spamreader:
             content=content.replace(">"+row[0]+"<",">"+row[1]+"<")

    print content

4.3 Setup 3rd party JS libraries on new site

Install syntaxhighlighter and google analytics, insert below code into conf.py:


BODY_END = """
<script type='text/javascript' src='//cdnjs.cloudflare.com/ajax/libs/SyntaxHighlighter/3.0.83/scripts/shCore.min.js'></script>
<script type='text/javascript' src='//cdnjs.cloudflare.com/ajax/libs/SyntaxHighlighter/3.0.83/scripts/shAutoloader.min.js'></script>
<script type='text/javascript'>
function path()
{
  var args = arguments,
      result = [];

  for(var i = 0; i < args.length; i++){
    result.push(args[i].replace('@', '//cdnjs.cloudflare.com/ajax/libs/SyntaxHighlighter/3.0.83/scripts/'));
  }

  return result;
};

SyntaxHighlighter.autoloader.apply(null, path(
  'applescript            @shBrushAppleScript.min.js',
  'actionscript3 as3      @shBrushAS3.min.js',
  'bash shell             @shBrushBash.min.js',
  'coldfusion cf          @shBrushColdFusion.min.js',
  'cpp c c++              @shBrushCpp.min.js',
  'clojure                @shBrushScala.min.js',
  'c# c-sharp csharp      @shBrushCSharp.min.js',
  'css                    @shBrushCss.min.js',
  'delphi pascal          @shBrushDelphi.min.js',
  'diff patch pas         @shBrushDiff.min.js',
  'erl erlang             @shBrushErlang.min.js',
  'groovy                 @shBrushGroovy.min.js',
  'java                   @shBrushJava.min.js',
  'jfx javafx             @shBrushJavaFX.min.js',
  'js jscript javascript  @shBrushJScript.min.js',
  'perl pl                @shBrushPerl.min.js',
  'php                    @shBrushPhp.min.js',
  'text plain             @shBrushPlain.min.js',
  'py python              @shBrushPython.min.js',
  'ruby rails ror rb      @shBrushRuby.min.js',
  'sass scss              @shBrushSass.min.js',
  'scala                  @shBrushScala.min.js',
  'sql                    @shBrushSql.min.js',
  'vb vbnet               @shBrushVb.min.js',
  'xml xhtml xslt html    @shBrushXml.min.js'
));
SyntaxHighlighter.all();
</script>
<script type='text/javascript'>
var _gaq = _gaq || [];

_gaq.push(['_setAccount', 'UA-29850823-2']);
_gaq.push(['_addDevId', 'i9k95']); // Google Analyticator App ID with Google
_gaq.push(['_trackPageview']);

(function() {
  var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
  ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
  var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
})();
</script>
"""

EXTRA_HEAD_DATA = """
<link rel='stylesheet' type='text/css' href='//cdnjs.cloudflare.com/ajax/libs/SyntaxHighlighter/3.0.83/styles/shCore.min.css'>
<link rel='stylesheet' type='text/css' href='//cdnjs.cloudflare.com/ajax/libs/SyntaxHighlighter/3.0.83/styles/shCoreEmacs.css'>
"""

In above code, shCoreEmacs.css is the color theme for syntaxhighlighter.

4.4 Upload

I use FileZilla as a FTP client. NcFTP is a command line alternative.

The best way to upload HTML files is Git.

Upload emacs package to marmalade

Thanks to Sebastian Wiesner for providing a python script to upload package to http://marmalade-repo.org/.


sudo pip install requests keyring

curl -L https://raw.githubusercontent.com/lunaryorn/dotfiles/master/emacs/bin/marmalade-upload > ~/bin/marmalade-upload && chmod +x ~/bin/marmalade-upload

~/bin/marmalade-upload -h # see help

Please note if the same version already exists on server, I need delete it before uploading.

Debug Emacs Lisp code the hard way

I met some issue when using helm several days ago.

With the help from Michael Heerdegen, I can locate the line error message is dumped.

But the problem is I cannot get the backtrace at all.

So I insert some backtrace printing code above that line and "M-x eval-buffer":


(with-output-to-temp-buffer "backtrace-output"
  (let ((var 1))
    (save-excursion
      (setq var (eval '(progn
                         (1+ var)
                         (list 'testing (backtrace))))))))

It turns out that some third party package I installed changes the emacs variable "display-buffer-function". I guess that's why the backtrace is blocked. Because backtrace is usually dumped into a backtrace buffer. And backtrace buffer's is actually influenced by the variable.

How to use ctags in Emacs effectively

Exuberant Ctags is a code navigation tool. It supports many language and could be integrated into Emacs well.

Please read EmacsWiki for basic usage.

I will talk about how I manage my ctags.

Basically ctags will produce a index file with file name TAGS. The full path of TAGS will be stored in a global list "tags-table-list".

An example of tags-table-list:

(setq tags-table-list '("~/wxWidgets-master/TAGS" "~/projs/Loris/src/desktop/TAGS"))

Every time we you "M-x find-tag", the TAGS file in above list will be read from the scratch to locate the definition of the symbol under cursor.

Here is my strategy to manage TAGS automatically:

  • I hard coded full path of TAGS in .emacs because I usually don't change project path.
  • In major mode hook like c++-mode-hook or js2-mode-hook I will check the directory path of current file. If it contains certain string, I suppose the file belong to certain project.
  • Then I will create TAGS for that project if needed
  • Every time when I save the file, I may update TAGS according to the value of tags-table-list.

Here is the code:

(defun my-project-name-contains-substring (REGEX)
  (let ((dir (if (buffer-file-name)
                 (file-name-directory (buffer-file-name))
               "")))
    (string-match-p REGEX dir)))

(defun my-create-tags-if-needed (SRC-DIR &optional FORCE)
  "return the full path of tags file"
  (let ((dir (file-name-as-directory (file-truename SRC-DIR)) )
       file)
    (setq file (concat dir "TAGS"))
    (when (or FORCE (not (file-exists-p file)))
      (message "Creating TAGS in %s ..." dir)
      (shell-command
       (format "ctags -f %s -e -R %s" file dir))
      )
    file
    ))

(defvar my-tags-updated-time nil)

(defun my-update-tags ()
  (interactive)
  "check the tags in tags-table-list and re-create it"
  (dolist (tag tags-table-list)
    (my-create-tags-if-needed (file-name-directory tag) t)
    ))

(defun my-auto-update-tags-when-save ()
  (interactive)
  (cond
   ((not my-tags-updated-time)
    (setq my-tags-updated-time (current-time)))
   ((< (- (float-time (current-time)) (float-time my-tags-updated-time)) 300)
    ;; < 300 seconds
    ;; do nothing
    )
   (t
    (setq my-tags-updated-time (current-time))
    (my-update-tags)
    (message "updated tags after %d seconds." (- (float-time (current-time))  (float-time my-tags-updated-time)))
    )
   ))

(defun my-setup-develop-environment ()
    (when (my-project-name-contains-substring "Loris")
      (cond
       ((my-project-name-contains-substring "src/desktop")
        ;; C++ project don't need html tags
        (setq tags-table-list (list
                               (my-create-tags-if-needed
                                (concat (file-name-as-directory (getenv "WXWIN")) "include"))
                               (my-create-tags-if-needed "~/projs/Loris/loris/src/desktop")))
        )
       ((my-project-name-contains-substring "src/html")
        ;; html project donot need C++ tags
        (setq tags-table-list (list (my-create-tags-if-needed "~/projs/Loris/loris/src/html")))
        ))))

(add-hook 'after-save-hook 'my-auto-update-tags-when-save)
(add-hook 'js2-mode-hook 'my-setup-develop-environment)
(add-hook 'web-mode-hook 'my-setup-develop-environment)
(add-hook 'c++-mode-hook 'my-setup-develop-environment)
(add-hook 'c-mode-hook 'my-setup-develop-environment)

UPDATE: There is some discussion at Google Plus about using ctags. Kaushal Modi recommended three emacs plugins:

  1. ctags-update
  2. etags-table
  3. etags-select

I tried these three plugins. ctags-update and etags-table duplicate my above elisp code. I prefer my own code because it's simpler and totally controllable. For example, the fact that I need only care about only one global variable tags-table-list makes my code shorter.

But I do like etags-select, it provide better UI for finding tag and I will use it from now on.

How to adapt my work style to the employer's standard in Emacs

I write some set up code in ~/.custom.el. It will detect which machine and what directory I'm working on. Then I hook the set up code when major mode (js2-mode, web-mode, for example) loaded.

In my ~/.emacs,

(if (file-readable-p (expand-file-name "~/.custom.el"))
     (load-file (expand-file-name "~/.custom.el")))

Content of my ~/.custom.el:

;; (getenv "HOSTNAME") won't work because $HOSTNAME is not an env variable
;; (system-name) won't work because as Optus required, my /etc/hosts is changed
(defun my/at-office ()
  (interactive)
  (let ((my-hostname (with-temp-buffer
                       (shell-command "hostname" t)
                       (goto-char (point-max))
                       (delete-char -1)
                       (buffer-string))
                     ))
    (and (string= my-hostname "my-sydney-workpc")
         (not (or (string= my-hostname "sydneypc")
                  (string= my-hostname "ChenBinMacAir")
                  (string= my-hostname "eee")
                  )))
    ))

(defun my/use-office-style ()
  (interactive)
  (let ((dir (if (buffer-file-name)
                 (file-name-directory (buffer-file-name))
               "")))
    (string-match-p "CompanyProject" dir)
    ))

(defun my/setup-develop-environment ()
  (cond
   ((my/use-office-style)
    (message "Office code style!")
    (setq coffee-tab-width 4)
    (setq javascript-indent-level 4)
    (setq js-indent-level 4)
    (setq js2-basic-offset 4)
    (setq web-mode-indent-style 4))
   (t
    (message "My code style!")
    (setq coffee-tab-width 4)
    (setq javascript-indent-level 2)
    (setq js-indent-level 2)
    (setq js2-basic-offset 2)
    (setq web-mode-indent-style 2))
   ))

(add-hook 'js2-mode-hook 'my/setup-develop-environment)
(add-hook 'web-mode-hook 'my/setup-develop-environment)

My answer to "How do you use Emacs without a directory tree on the side of the editor?" on quora.com

This is original link of the question. Please note I focus on programming for large projects. I need search/grep/modify files scattered in different places without documentation.

If your use case is different, a embedded file explorer may be a better choice

Tools I use to replace a file explorer

  1. ido-find-file (emacs), just type any characters, it will fuzz search matched file in parent/sibling/current/or-whatever directory.
  2. helm-find-file (emacs), this one use regular expression and has bigger window
  3. recentf and helm (emacs), I use regular expression open recent opened files.
  4. ctags, gtags or whatever tag tools (CLI tool), as mentioned by other people
  5. I also use lots of bash functions written by myself, those functions are trivial, but combined with a wonderful tool called percol, they become really powerful.
  6. I use some CLI clipboard tool so I can easily share the file path between terminal/emacs/firefox.
  7. there is also a emacs bundled feature called speedbar, which is similar to the file explorer, I used it once, but it does not fit in my ninja style ;) It's fine but the UI is almost same to those average text editors.
  8. I also write some elisp snippets, for example, convert absolute path to relative path; given one relative path, output one absolute path, etc …
  9. there is also some git based emacs plugin: you can search file under the project root directory.
  10. there is a CLI tool called fasd which can let you interactively select the file or folder you recently visited.
  11. many other tools, plug-ins, code snippets I can use …

Use case

  1. I need search a big project for a certain library (it's a freemarker file I need include) whose full path I don't know, I just search the API's name by type:

"gef keyword *". gef is my bash function based on grep.

  1. matched files are displayed instantly (grep is much more quick than IDEs, BTW) in a interactive console filter (use percol), I type a regular expression to filter file name and press enter
  2. the full path is in clipboard now, then I type ":e Shift-insert" to open that file.

This is the most difficult case, I usually strike less keys and not get out of Emacs.

  1. If i need insert this file's relative path, I press a hot key and another emacs plugin will correctly convert absolute path to relative one (relative to the file I'm editing in Emacs) and insert it into my editor.

Demo

Here is screen cast. In this demo I grepped and open a ftl file and then search another js file in the same project. http://blog.binchen.org/wp-content/uploads/2014/02/wpid-emacs-grep-search-open-file.gif

One liner to download and install emacs lisp source code

Run the below command in shell,


curl http://ftp.gnu.org/gnu/emacs/emacs-`emacs --version|head -n1|sed 's/[^0-9.]*\([0-9]\+\.[0-9]\+\).*/\1/g'`.tar.gz | sudo tar --exclude=admin/* --exclude=etc/* --exclude=nt/* --exclude=doc/* --exclude=leim/* -C /usr/share/emacs/`emacs --version|head -n1|sed 's/[^0-9.]*\([0-9]\+\.[0-9]\+\).*/\1/g'`/lisp/ --strip-components=2 --wildcards -zxv *.el

For example, after installing the lisp code. You can M-x find-function package-list-packages in Emacs, you will jump to the definition of that command in "/usr/share/emacs/24.3/lisp/emacs-lisp/package.el" (I assume your emacs version is 24.3).

Add current directory into load-path in Emacs

Useful when developing emacs plugin:

(defun add-pwd-into-load-path ()
  "add current directory into load-path, useful for elisp developers"
  (interactive)
  (let ((dir (expand-file-name default-directory)))
    (if (not (memq dir load-path))
        (add-to-list 'load-path dir)
        )
    (message "Directory added into load-path:%s" dir)
    )
  )

Code Rush

小结,代码库庞大,质量有问题,招了一些废物.M Toy:"我们不是要痛扁谁…". 1998年3月31日,Netscape公开其源代码.

人名 背景
Jamie Zawinski 黑客,擅长C/Perl/Lisp,不喜欢C++,XEmacs开发者,Netscape早期版本开发者)解释源代码的意义.注意键盘符合人体工程学.
Michael Toy 牢游戏Rogue的开发者,该游戏被PC World杂志2009年评为史上最伟大PC游戏第六,是Diablo的老祖宗,Netscape最早员工
Jim Barksdale CEO
Jim Roskind Security Expert,开发大型软件,很麻烦.还说了一大堆屁话.
Scott Collins 值夜班,惨透了.
Brendan Eich javascript
Jeff Weinstein programmer at Netscape.并不是Netscape最优秀的员工.嘿嘿.
Tara Hernandez QA
Blake Ross http://www.insanely-great.com/news.php?id=3993

My answer to "What are some must-have emacs additions" on quora.com

This is my answer for What are some must-have additions to your .emacs file? on quora.com.

Emacs 24 is already bundled with a package manager. So you can download and install the package without leaving Emacs. See http://www.emacswiki.org/emacs/ELPA for how to set up.

If you want to know more tips about specific plugin. Please search its name in Emacswiki.

1 Warning

I'm a web programmer, so my focus could be different from other emacs users.

2 third party plugins

2.1 company-mode

auto completing code, it requires less key typing than another emacs plugin auto-complete. Besides, it's actively maintained

2.2 yasnippet

insert template

2.3 js2-mode

javascript development, best of the best

2.4 evil-mode

vim simulator, I cannot live without it as a vi guy

2.5 evil-leader

evil's plugin, simple and powerful if you realize that ALL the hot keys (for example, I map "M-x" to ",mx") can be re-mapped with it

2.6 evil-matchit (written by me)

jump between tag pair, port of vim matchit

2.7 evil-nerd-commenter (written by me)

(un)comment by lines, port of nerd-comment

2.8 helm

hard to describe but powerful

2.9 web-mode

major-mode for all html templates

2.10 windows-numbering.el

sub-window navigation, best one. well, it occupies hotkey from M-1 to M-9, For normal emacs user this could be issue. but I'm evil guy;)

2.11 smex

you don't need remember hot key or command any more

3 plugins bundled with emacs 24

3.1 org-mode

gtd tool, it changed my life, as git,vim,emacs did to me ages ago.

3.2 winner-mode

undo sub window-layout

3.3 gnus

hate and love it. the UI is too geek. but has some killer feature, for example, forward email, send email quickly, attach file. but the point is I can use with other awesome plugins like yasnippet, company-mode,evil.

3.4 ido

similar to helm, my tip, you can tell helm when to use ido, when to use helm.

3.5 imenu

display list of function in current buffer, the point is it is programmable

3.6 flymake

real time syntax check

3.7 flyspell

best spell checker, if you know how to do programming for it

4 The last and best plugin

Lisp!

  • simple
  • no learning curve,
  • it's syntax ensure that its plugin developer are experienced developers who knows at least three programming languages.

5 Tip

If I find some awesome plugin, I will investigate the original author and make sure what other tools he/she uses or develops

For example, mooz maintains js2-mode, he also developed:

keysnail
firefox is converted into Emacs, best one in similar tools (I tried 3 or 2 similar firefox/chrome addons before settled on keysnail)
percol
A genius utility for shell operation. Make me ten times faster when working in bash shell. Without it, I will quit my programming career immediately.
Share