blog.lobraun.de Raspberry Pi

Silent startup failure for MongoDB on Raspberry Pi (and other systems)

Yesterday I tried to install mongodb on my Raspberry Pi. Raspbian Jessie ships with a default mongo 2.4 package, so the installation should be pretty straightforward:

apt-get install mongodb

However, after the installation finished, the service did not start:

service mongodb start

did not result in a started mongod process. No log files in /var/log/mongodb were produced, indicating that the process stops before the logging is initialized. If the mongodb server is started manually, a very early crash can be observed:

sudo -u mongodb /usr/bin/mongod  --config /etc/mongodb.conf
Sun Mar 27 07:48:45.296 terminate() called, printing stack (if implemented for platform):
 0x664160 0x16d954 0x767ea9a0
 /usr/bin/mongod(_ZN5mongo15printStackTraceERSo+0x1c) [0x664160]
 /usr/bin/mongod(_ZN5mongo11myterminateEv+0x54) [0x16d954]
 /usr/lib/arm-linux-gnueabihf/libstdc++.so.6(+0x4a9a0) [0x767ea9a0]
Sun Mar 27 07:48:45.299 Got signal: 6 (Aborted).
 
Sun Mar 27 07:48:45.302 Backtrace:
0x664160 0x16e370 0x765be180 0x765bcf70
 /usr/bin/mongod(_ZN5mongo15printStackTraceERSo+0x1c) [0x664160]
 /usr/bin/mongod(_ZN5mongo10abruptQuitEi+0x2a0) [0x16e370]
 /lib/arm-linux-gnueabihf/libc.so.6(__default_sa_restorer_v2+0) [0x765be180]
 /lib/arm-linux-gnueabihf/libc.so.6(gsignal+0x38) [0x765bcf70]

Stracing the syscalls shows that the error is generated shortly after mongod tries to load locales:

munmap(0x76f24000, 4096)                = 0
open("/usr/lib/locale/en_GB.UTF-8/LC_NUMERIC", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/usr/lib/locale/en_GB.utf8/LC_NUMERIC", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/usr/lib/locale/en_GB/LC_NUMERIC", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/usr/lib/locale/en.UTF-8/LC_NUMERIC", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/usr/lib/locale/en.utf8/LC_NUMERIC", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/usr/lib/locale/en/LC_NUMERIC", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
gettimeofday({1459065376, 950211}, NULL) = 0
open("/etc/localtime", O_RDONLY|O_CLOEXEC) = 4
fstat64(4, {st_mode=S_IFREG|0644, st_size=118, ...}) = 0
fstat64(4, {st_mode=S_IFREG|0644, st_size=118, ...}) = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x76f24000
read(4, "TZif2\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\1\0\0\0\1\0\0\0\0"..., 4096) = 118
_llseek(4, -6, [112], SEEK_CUR)         = 0
read(4, "\nUTC0\n", 4096)               = 6
close(4)                                = 0
munmap(0x76f24000, 4096)                = 0
fstat64(1, {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 1), ...}) = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x76f24000
write(1, "Sun Mar 27 07:56:16.950 ", 24Sun Mar 27 07:56:16.950 ) = 24
write(1, "terminate() called, printing sta"..., 65terminate() called, printing stack (if implemented for platform):) = 65
write(1, "\n", 1
)                       = 1
futex(0x7667e5d8, FUTEX_WAKE_PRIVATE, 2147483647) = 0
write(1, "0x664160 0x16d954 0x7679a9a0 \n", 300x664160 0x16d954 0x7679a9a0
) = 30
write(1, " /usr/bin/mongod(_ZN5mongo15prin"..., 65 /usr/bin/mongod(_ZN5mongo15printStackTraceERSo+0x1c) [0x664160]
) = 65
write(1, " /usr/bin/mongod(_ZN5mongo11myte"..., 59 /usr/bin/mongod(_ZN5mongo11myterminateEv+0x54) [0x16d954]
) = 59
write(1, " /usr/lib/arm-linux-gnueabihf/li"..., 68 /usr/lib/arm-linux-gnueabihf/libstdc++.so.6(+0x4a9a0) [0x7679a9a0]
) = 68

It turns out that I did not change the default locale on my installation (which was en_GB.UTF-8), and did not make sure a proper locale was generated for this setting.

To fix the error, first make sure you pick the right locale. This can be changed by running

dpkg-reconfigure locales

There, you can first pick the locales that should be generated:

Afterwards, you can pick the locale that should be used on the system:

dpkg-reconfigure will then generate the locales that are supported. The changes will be applied after the next reboot. Make sure they are actually used in your current session by running

export LANG=*<YOUR_LOCALE_SELECTION>*

Afterwards, mongodb should start just fine.

If you are not on a debian system, you can make sure that the proper locales are generated by editing

/etc/locale.gen

and uncommenting all locales that you need. Afterwards you need to generate

locale-gen

which will generate the locale files. Setting the LANG environment variable is something that is distribution specific. Check your manual on where you can find the proper place to configure the locale. On debian-based systems, you can find it in

/etc/default/locale

on Gentoo, check your

/etc/env.d/02locale

Mount SD Cards within VirtualBox on Mac OS X

Sometimes you need to mount your SD cards inside a VirtualBox machine to work with the files on the card. Working with the Raspberry Pi SD card on Mac OS X is an example for this.

My Raspberry Pi runs on Linux with some version of the EXT file system. While there is some support for this on Mac OS X, the available solutions have a lot of limitations, especially when it comes to write support on EXT file systems. As I do not have an USB keyboard, I sometimes need to mount the filesystem on another system to fix a startup problem when some of my experiments go wrong.

VirtualBox in theory allows for passing the internal card reader of the MacBooks to the virtual machine. Unfortunately, this does not work for me (and it seems for many other people).

What you can do instead is to create a virtual image, that passes control to the device node to the VM. The following steps will show you how to accomplish this. I put together a script that performs these steps for you. You can download the script here:

Insert SD Cards into your card reader

Insert the card into the SD card reader slot.

Unmount the partitions from Mac OS X

You can either do this using the Disk Utility:

Be sure that you choose Unmount instead of Eject for all the mounted partitions on your SD card.

Alternatively, you can unmount it using the command line. First, find the name of your SD Card running the command

diskutil list

You should get a list of all the disks and partitions on your system. After you identified the proper disk, e.h. /dev/disk2 on my system, just run the command

diskutil unmountDisk /dev/diskX

where diskX is your disk.

Create a VDMK file that maps to the disk

Afterwards, create a VMDK file that maps to the disk:

sudo VBoxManage internalcommands createrawvmdk -filename <vmdk_file> -rawdisk </dev/diskX>

where vmdk_file is the filename that should be created and /dev/diskX is the name of the SD card.

Grant permissions on these devices

As you will probably run VirtualBox as your normal user, you should make sure that he has the rights to perform all operations on the device. The disk device usually belongs to root:operator, and your user is probably not part of this group. So in order to get full access to the device, you need to either get to be a member of the operator group (and give full access to the group), or you give all access permissions to all users on the machine:

sudo chmod 666 vmdk_file
sudo chmod 666 /dev/diskX
Add this VDMK file to the virtual machine that should get access to the SD card reader

You can either do this using the VirtualBox graphical user interface. Select the machine that should work with the SD card. Open the settings dialog and add the file (in this example: sdcard.vmdk) as a hard disk to the SATA controller:

Afterwards, you can start the virtual machine and you should have access to the card inside the VM.

Instead of using the VirtualBox GUI, you can also add the disk to the VM on the command line:

VBoxManage storageattach "<name_of_your_vm>" --medium <vmdk_file> --storagectl SATA --port <port_number> --type hdd

Where name_of_your_vm is the name of the VM that should get access to the SD card, vmdk_file is the file that you just created, and port_number is the port of your SATA device that your file should be attached to.

I put together a script that does these steps for you. You can download the script here.



Start the virtual machine

When you start the VM, you might get the following error message:

This probably comes from the fact that OS X remounted one or more of the SD card partitions. Try to unmount them and again and start the virtual machine again.

If you have an idea how this can be achieved more easily, please let me know in the comments.

Create a TimeMachine Backup Storage on the Raspberry Pi

[TimeMachine](http://en.wikipedia.org/wiki/Time_Machine_(OS_X)) is the way to backup your data on Mac systems. The backup and restore procedure work nicely out of the box with very little hassle involved. I used to backup my data on a local USB disk for some years. However, backups to a USB disk require you to connect the disk, otherwise no backups will be performed.

Apple will sell you the Airport Time Capsulate, which allows for remote backups. But it is also possible to configure your Raspberry Pi to offer remote backup capabilities for the TimeMachine. Turning your Raspberry into a backup target for the TimeMachine is as simple as installing an AFP server on it. In this blog post, I'll walk through the process of configuring your raspberry to be a remote target for TimeMachine backups.

Prerequisites

You need the following things to get started:

  • Raspberry Pi
  • SD card
  • storage device for the backups, e.g. a USB hard disk
  • Raspbian OS image for the Pi
  • network connection to your local network

Install Raspbian on the device and connect it to your network. There are many installation guides for installing the Raspbian image out there. Since I assume that you are using a Mac, you might want to have a look at this guide for information on how to install Raspbian to the SD card of your Raspberry Pi.

After connecting the device to your home network, you should be able to connect to your device via its assigned IP address. You should assign a static IP address to the device, or make sure that the device has a internal DNS name in your home network. If you have a FRITZ!Box in your home network, then this article can tell you how to setup this for your Raspberry.

Configuring the Network Shares

The TimeCapsulate functionality requires that the Raspberry provides network shares over the AFP protocol.

The protocol is a proprietary file protocol that has been around on Mac systems for quite some time. There is a free open source implementation of the protocol which is provided by Netatalk. This server allows you to create a file server that can be used as a target for your TimeMachine backups.



Installing Netatalk

As always with Raspbian, installing software is easy. Run

sudo apt-get install netatalk

to install the package. Then make it available at boot time by running

sudo update-rc.d netatalk defaults

Configuring Netatalk

There are a number of configurations that you need to adopt. First, choose a directory on your USB drive that you want to use as a TimeMachine target. In this example, we assume that you want to store your backups at

/mnt/timemachine/

The first file you need to edit is

/etc/netatalk/AppleVolumes.default

You can first delete the sharing of the Raspberry's users home directories (if you want to). Identify and delete the line

~/                      "Home Directory"

in the config file. Remove it and replace it with your upcoming backup location. For this, you need to decide

  • who should be allowed to access the share, i.e. which user
  • a fancy name for the share
  • the size of the share, i.e. how much backup space you want to allocate to the TimeMachine. This can be handy if you want to use the disk for other data as well.

Using this information, you can create your config line:

/mnt/timemachine "\<fancy_name_for_share\>" allow:<user> cnidscheme:dbd options:upriv,usedots,tm volsizelimit:<size_in_megabytes>

For example, you can configure the AFP share as

 /mnt/timemachine "My_MacBook" allow:pi cnidscheme:dbd options:upriv,usedots,tm volsizelimit:100000

for a volume called My_MacBook that is accessible by the user pi and can take up to 100GB of disk space.

Note that the user that you define in the allow section must be a valid user on your Raspberry Pi and needs write permissions on the directory /mnt/timemachine.

Now restart the Netatalk service to make the new configuration available:

service netatalk restart



Configuring the TimeMachine

Start the TimeMachine and choose to configure your backup.

Click the button Select Backup Disk .... You should see your AFP with the previously defined share name:

Select your share and tick the check box Encrypt Backups (recommended). Enter your credentials after selecting the disk. Your MacBook should now start to perform the backup to the Raspberry.

Raspberry Pi: USB 3.0 Disks do not appear as device nodes

The Raspberry is a nice cheap platform that can be used for many different purposes. However, the fact that it is a cheap platform means that all kinds of hardware-related problems can occur. One common problem is related to USB devices.

If you connect to many devices via hubs, or devices that have high power requirements, then these devices might not work properly. As always, there is good documentation, which can help you to determine the problem.

One thing that is not included, is a tip on what to do if you have a problem. In my case, I attached a USB 3.0 disk to the Raspberry. The disk was detected and the kernel produced the following output:

[    3.838192] usb 1-1.4: new high-speed USB device number 4 using dwc_otg
[    3.949172] usb 1-1.4: New USB device found, idVendor=04c5, idProduct=2028
[    3.958122] usb 1-1.4: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[    3.958133] usb 1-1.4: Product: MB86C311
[    3.958143] usb 1-1.4: Manufacturer: FUJITSU
[    3.958151] usb 1-1.4: SerialNumber: 201105303687
[    3.959256] usb-storage 1-1.4:1.0: USB Mass Storage device detected

The command lsusb showed that the device was properly detected:

/:  Bus 01.Port 1: Dev 1, Class=root_hub, Driver=dwc_otg/1p, 480M
|__ Port 1: Dev 2, If 0, Class=hub, Driver=hub/5p, 480M
    |__ Port 1: Dev 3, If 0, Class=vend., Driver=smsc95xx, 480M
    |__ Port 4: Dev 4, If 0, Class=stor., Driver=usb-storage, 480M

However, the kernel did not create a proper device node in /dev/ that could be used to access the filesystem on the device. It turned out that this is one of the power issues.

According to the Internet the USB ports are powered with 600mA with the default settings. Certain USB devices, such as my disk, require more power to work properly. It is possible to increase the current on the USB ports to 1200mA by adding

max_usb_current=1

to /boot/config.txt and rebooting the device. This requires that your power supply supports this increase, which my power supply does.

The device will then be properly detected after a reboot, and device nodes will be created.