blog.lobraun.de Mac OS X

Python PIP Package Updates on Mac OS X

Updating all python packages that are installed via pip is often referred to as running something like the commands

pip freeze --local | grep -v '^\-e' | cut -d = -f 1  | xargs -n1 pip install -U

However, some problems can occur on Mac OS X, as certain default packages are included into the base python installation.

Packages that rely on new versions of pre-installed packages will throw errors as the default python search path prefers the pre-installed packages. An example is matplotlib: Version 1.5 of matplotlib requires a newer version of the six package than version 1.4.1 that is shipped on El Capitan and errors out with the following message:

/Library/Python/2.7/site-packages/dateutil/rrule.py in <module>()
 14 
 15 from six import advance_iterator, integer_types
---> 16 from six.moves import _thread
 17 
 18 __all__ = ["rrule", "rruleset", "rrulestr",

ImportError: cannot import name _thread

You can fix this problem by properly setting your python path environment to prefer the python pip installed packages. The path can for example be set in the ~/.bashrc to point to

export PYTHONPATH="/Library/Python/2.7/site-packages:$PYTHONPATH"

PEP8 for vim using flake8

In order to auto format your python code according to the guidelines defined by PEP8, you can use some sane default for editing python files and flake8 to check and fix possible errors in your scripts.

Default indentation rules for python files

The default indentation rules do not comply with PEP8. In order to change them for python files, create a new file in ~/.vim/ftplugin/python.vim and place the following content in this file:

setlocal tabstop=4
setlocal softtabstop=4
setlocal shiftwidth=4
setlocal smarttab
setlocal expandtab
setlocal shiftround
setlocal autoindent

PEP8 recommends that code should have a maximum line length of 79 characters. You can enforce this by adding the following line to the file:

setlocal textwidth=79

However, it can be a good idea to have longer lines, I prefer to not have a hard wrap at 79. You can therefore opt to show a marker that helps you to avoid the character line limit by setting:

setlocal colorcolumn=80
Install flake8

flake8 is a python module that you can install using your favourite package manager, e.g. using pip:

sudo pip install flake
Install vim-pathogen and vim-flake8

vim-pathogen is a vim script that helps with managing the runtimepath to simplify the vim plugin management. You can install it by running the following command:

mkdir -p ~/.vim/autoload ~/.vim/bundle && \
curl -LSso ~/.vim/autoload/pathogen.vim https://tpo.pe/pathogen.vim

then add the following to your .vimrc

execute pathogen#infect()

Afterwards, install vim-flake8:

cd ~/.vim/bundle
git clone https://github.com/nvie/vim-flake8

This is everything you need to do. Afterwards, you can open vim on a python file and press < F7 > to start flake and obtain information on your formatting.

Mount SD Cards within VirtualBox on Mac OS X

Sometimes you need to mount your SD cards inside a VirtualBox machine to work with the files on the card. Working with the Raspberry Pi SD card on Mac OS X is an example for this.

My Raspberry Pi runs on Linux with some version of the EXT file system. While there is some support for this on Mac OS X, the available solutions have a lot of limitations, especially when it comes to write support on EXT file systems. As I do not have an USB keyboard, I sometimes need to mount the filesystem on another system to fix a startup problem when some of my experiments go wrong.

VirtualBox in theory allows for passing the internal card reader of the MacBooks to the virtual machine. Unfortunately, this does not work for me (and it seems for many other people).

What you can do instead is to create a virtual image, that passes control to the device node to the VM. The following steps will show you how to accomplish this. I put together a script that performs these steps for you. You can download the script here:

Insert SD Cards into your card reader

Insert the card into the SD card reader slot.

Unmount the partitions from Mac OS X

You can either do this using the Disk Utility:

Be sure that you choose Unmount instead of Eject for all the mounted partitions on your SD card.

Alternatively, you can unmount it using the command line. First, find the name of your SD Card running the command

diskutil list

You should get a list of all the disks and partitions on your system. After you identified the proper disk, e.h. /dev/disk2 on my system, just run the command

diskutil unmountDisk /dev/diskX

where diskX is your disk.

Create a VDMK file that maps to the disk

Afterwards, create a VMDK file that maps to the disk:

sudo VBoxManage internalcommands createrawvmdk -filename <vmdk_file> -rawdisk </dev/diskX>

where vmdk_file is the filename that should be created and /dev/diskX is the name of the SD card.

Grant permissions on these devices

As you will probably run VirtualBox as your normal user, you should make sure that he has the rights to perform all operations on the device. The disk device usually belongs to root:operator, and your user is probably not part of this group. So in order to get full access to the device, you need to either get to be a member of the operator group (and give full access to the group), or you give all access permissions to all users on the machine:

sudo chmod 666 vmdk_file
sudo chmod 666 /dev/diskX
Add this VDMK file to the virtual machine that should get access to the SD card reader

You can either do this using the VirtualBox graphical user interface. Select the machine that should work with the SD card. Open the settings dialog and add the file (in this example: sdcard.vmdk) as a hard disk to the SATA controller:

Afterwards, you can start the virtual machine and you should have access to the card inside the VM.

Instead of using the VirtualBox GUI, you can also add the disk to the VM on the command line:

VBoxManage storageattach "<name_of_your_vm>" --medium <vmdk_file> --storagectl SATA --port <port_number> --type hdd

Where name_of_your_vm is the name of the VM that should get access to the SD card, vmdk_file is the file that you just created, and port_number is the port of your SATA device that your file should be attached to.

I put together a script that does these steps for you. You can download the script here.



Start the virtual machine

When you start the VM, you might get the following error message:

This probably comes from the fact that OS X remounted one or more of the SD card partitions. Try to unmount them and again and start the virtual machine again.

If you have an idea how this can be achieved more easily, please let me know in the comments.