blog.lobraun.de Mostly technical stuff

ELPY Cheat Sheet

ELPY is a python programming environment. The features are documented here.

The following table summarises the default key bindings for the functions of version 1.11.0:

Key Binding Function
C-down Forward one indentation block
C-up Backward one indentation block
C-left Backward one indentation level
C-right Forward one indentation level
M-down Move line or region down
M-up Move line or region up
M-left Move line or indentation block left
M-right Move line or indentation block right
M-x elpy-set-project-root Set the root directory of the python project
C-c C-f Find file in project
C-c C-s Regular expression match in project
M-TAB Complete current statement
M-. Goto defition
M-* Return to position from last definition lookup
C-c C-z Switch to python shell
C-c C-c Send python code to shell (active region or complete buffer)
C-c RET Send current line to python shell
C-M-x Sends code of current top level function to python shell
C-c C-v Syntax check with flake8
C-c C-n Next flake8 error
C-c C-p Previous flake8 error
C-c C-t Start tests
C-c C-e Edit all occurrences of the symbol at point at once
C-c C-r f Format code using the available formatter
C-c C-r i Query for new imports of unresolved symbols, and remove unreferenced imports
C-c C-r r Run the Elpy refactoring interface for Python code

Python PIP Package Updates on Mac OS X

Updating all python packages that are installed via pip is often referred to as running something like the commands

pip freeze --local | grep -v '^\-e' | cut -d = -f 1  | xargs -n1 pip install -U

However, some problems can occur on Mac OS X, as certain default packages are included into the base python installation.

Packages that rely on new versions of pre-installed packages will throw errors as the default python search path prefers the pre-installed packages. An example is matplotlib: Version 1.5 of matplotlib requires a newer version of the six package than version 1.4.1 that is shipped on El Capitan and errors out with the following message:

/Library/Python/2.7/site-packages/dateutil/rrule.py in <module>()
 14 
 15 from six import advance_iterator, integer_types
---> 16 from six.moves import _thread
 17 
 18 __all__ = ["rrule", "rruleset", "rrulestr",

ImportError: cannot import name _thread

You can fix this problem by properly setting your python path environment to prefer the python pip installed packages. The path can for example be set in the ~/.bashrc to point to

export PYTHONPATH="/Library/Python/2.7/site-packages:$PYTHONPATH"

PEP8 for vim using flake8

In order to auto format your python code according to the guidelines defined by PEP8, you can use some sane default for editing python files and flake8 to check and fix possible errors in your scripts.

Default indentation rules for python files

The default indentation rules do not comply with PEP8. In order to change them for python files, create a new file in ~/.vim/ftplugin/python.vim and place the following content in this file:

setlocal tabstop=4
setlocal softtabstop=4
setlocal shiftwidth=4
setlocal smarttab
setlocal expandtab
setlocal shiftround
setlocal autoindent

PEP8 recommends that code should have a maximum line length of 79 characters. You can enforce this by adding the following line to the file:

setlocal textwidth=79

However, it can be a good idea to have longer lines, I prefer to not have a hard wrap at 79. You can therefore opt to show a marker that helps you to avoid the character line limit by setting:

setlocal colorcolumn=80
Install flake8

flake8 is a python module that you can install using your favourite package manager, e.g. using pip:

sudo pip install flake
Install vim-pathogen and vim-flake8

vim-pathogen is a vim script that helps with managing the runtimepath to simplify the vim plugin management. You can install it by running the following command:

mkdir -p ~/.vim/autoload ~/.vim/bundle && \
curl -LSso ~/.vim/autoload/pathogen.vim https://tpo.pe/pathogen.vim

then add the following to your .vimrc

execute pathogen#infect()

Afterwards, install vim-flake8:

cd ~/.vim/bundle
git clone https://github.com/nvie/vim-flake8

This is everything you need to do. Afterwards, you can open vim on a python file and press < F7 > to start flake and obtain information on your formatting.

Mount SD Cards within VirtualBox on Mac OS X

Sometimes you need to mount your SD cards inside a VirtualBox machine to work with the files on the card. Working with the Raspberry Pi SD card on Mac OS X is an example for this.

My Raspberry Pi runs on Linux with some version of the EXT file system. While there is some support for this on Mac OS X, the available solutions have a lot of limitations, especially when it comes to write support on EXT file systems. As I do not have an USB keyboard, I sometimes need to mount the filesystem on another system to fix a startup problem when some of my experiments go wrong.

VirtualBox in theory allows for passing the internal card reader of the MacBooks to the virtual machine. Unfortunately, this does not work for me (and it seems for many other people).

What you can do instead is to create a virtual image, that passes control to the device node to the VM. The following steps will show you how to accomplish this. I put together a script that performs these steps for you. You can download the script here:

Insert SD Cards into your card reader

Insert the card into the SD card reader slot.

Unmount the partitions from Mac OS X

You can either do this using the Disk Utility:

Be sure that you choose Unmount instead of Eject for all the mounted partitions on your SD card.

Alternatively, you can unmount it using the command line. First, find the name of your SD Card running the command

diskutil list

You should get a list of all the disks and partitions on your system. After you identified the proper disk, e.h. /dev/disk2 on my system, just run the command

diskutil unmountDisk /dev/diskX

where diskX is your disk.

Create a VDMK file that maps to the disk

Afterwards, create a VMDK file that maps to the disk:

sudo VBoxManage internalcommands createrawvmdk -filename <vmdk_file> -rawdisk </dev/diskX>

where vmdk_file is the filename that should be created and /dev/diskX is the name of the SD card.

Grant permissions on these devices

As you will probably run VirtualBox as your normal user, you should make sure that he has the rights to perform all operations on the device. The disk device usually belongs to root:operator, and your user is probably not part of this group. So in order to get full access to the device, you need to either get to be a member of the operator group (and give full access to the group), or you give all access permissions to all users on the machine:

sudo chmod 666 vmdk_file
sudo chmod 666 /dev/diskX
Add this VDMK file to the virtual machine that should get access to the SD card reader

You can either do this using the VirtualBox graphical user interface. Select the machine that should work with the SD card. Open the settings dialog and add the file (in this example: sdcard.vmdk) as a hard disk to the SATA controller:

Afterwards, you can start the virtual machine and you should have access to the card inside the VM.

Instead of using the VirtualBox GUI, you can also add the disk to the VM on the command line:

VBoxManage storageattach "<name_of_your_vm>" --medium <vmdk_file> --storagectl SATA --port <port_number> --type hdd

Where name_of_your_vm is the name of the VM that should get access to the SD card, vmdk_file is the file that you just created, and port_number is the port of your SATA device that your file should be attached to.

I put together a script that does these steps for you. You can download the script here.



Start the virtual machine

When you start the VM, you might get the following error message:

This probably comes from the fact that OS X remounted one or more of the SD card partitions. Try to unmount them and again and start the virtual machine again.

If you have an idea how this can be achieved more easily, please let me know in the comments.

Ghost Blogging: Move from SQLite to Postgres or MySQL

After moving to the ghost blogging platform, I started running the system with SQLite as backend.

Now I wanted to move from SQLite to Postgres. A Google search brought up articles like this one, which try to migrate the content of the database. Scripts are used to dump the SQLite content to an export file format, to change the SQL syntax from SQLite to Postgres and to import it to the database. This has the advantage that all your settings, especially the global ones, will be converted. If that is not necessary for you, then you can use the export/import function of Ghost.

This function allows to migrate the content without having to touch the database. However, this way will loose your global settings.

Export your Blog data

Ghost ships a way to export your data. In the management interface, navigate to Settings -> Lab. There you can find the Export button:

With this function, you can export your blog content into a JSON file. This file can later be imported to the system, after you reconfigured it to use a Postgres backend.

Reconfigure the backend

At first, you need to setup your postgres database. Consult your favourite source of documentation for how to do this.

Now create a user for your blog inside the psql command line:

CREATE USER <blog_user> WITH PASSWORD '<blog_password>'; 

now create your database:

CREATE DATABASE <your_db_name> OWNER <blog_user>;

You can also give all privileges to this database to an existing user:

GRANT ALL PRIVILEGES ON <your_db_name> TO <blog_user>;

Reconfigure the Ghost backend

Go to your blog directory and edit the file config.js, according to the documentation. Remove your old db confug and import the new one:

database: {
    client: 'postgres',
    connection: {
      host: '127.0.0.1',
      user: 'username',
      password: 'password',
      database: 'databasename',
      port: '5432'
    },

Now you can restart ghost so that it uses the new backend.

Import your blog data

You are asked to setup your blog. Enter your blog description. After entering the right information, you can then again navigate to Settings -> Lab and import the blog data that you exported in the previous step.

Unfortunately, you have to recreate your global settings with this way to migrate your DB.

Newer Posts Older Posts