Blog pro de Jean-Baptiste HEREN - Linux

Convert virtual drive from virtualbox to vmware

Jean-Baptiste HEREN — Tue, 21 Jun 2011 23:03:00 +0200

You have a virtualbox image and want to use it in VMWARE ?

If the .ova file does not fits your needs, you can try to convert the hard drive and then use it directly in a configured vmware machine.

VirtualBox provides a command line tool to convert from .vdi to .vmdk : VBoxManage clonehd

Here is an Example

[script]
jb$ cd /Users/jb/VirtualBox VMs/myvirtualdrive-bash: cd: /Users/jb/VirtualBox: No such file or directory
jb$ cd "/Users/jb/VirtualBox VMs/myvirtualdrive"
jb$ ls
Logs                myvirtualdrive.vbox      myvirtualdrive.vbox-prev myvirtualdrive.vdi
jb$ VBoxManage clonehd myvirtualdrive.vdi --format VMDK myvirtualdrive.vmdk
0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100%
Clone hard disk created in format 'VMDK'. UUID: xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx

Documentation from VirtualBox

This command duplicates a registered virtual hard disk image to a new image file with a new unique identifier (UUID). The new image can be transferred to another host system or imported into VirtualBox again using the Virtual Media Manager; see the section called “The Virtual Media Manager” and the section called “Cloning disk images”. The syntax is as follows:

VBoxManage clonehd         <uuid>|<filename> <outputfile>
                           [--format VDI|VMDK|VHD|RAW|<other>]
                           [--variant Standard,Fixed,Split2G,Stream,ESX]
                           [--existing]

where the parameters mean:

format Allow to choose a file format for the output file different from the file format of the input file.

variant Allow to choose a file format variant for the output file. It is a comma-separated list of variant flags. Not all combinations are supported, and specifying inconsistent flags will result in an error message.

existing Perform the clone operation to an already existing destination medium. Only the portion of the source medium which fits into the destination medium is copied. This means if the destination medium is smaller than the source only a part of it is copied, and if the destination medium is larger than the source the remaining part of the destination medium is unchanged.

Repair HFS+ Volume on OSX

Jean-Baptiste HEREN — Wed, 09 Feb 2011 00:26:00 +0100

Yesterday I had a little surprise on one of my disks (the backup one.Disk utility was unable to repair it, but here you will find some tips

1- read your disk with Linux

Boot on Ubuntu live CD and try to open your disk. It will mount read-only (default for HFS) but you will be able to grab most of your files to a network or FAT external drive. This saved my life a couple of times.

2- repair with fsck on OSX

Fsck is also available on OSX. Let's open your terminal. On the following commands, replace /dev/disk1s2 with the location of your disk's partition.

2.1 - run a diagnostc

[script]
MyMac:~ jb$ fsck -fy /dev/disk1s2
** /dev/rdisk1s2
BAD SUPER BLOCK: MAGIC NUMBER WRONG

LOOK FOR ALTERNATE SUPERBLOCKS? yes

SEARCH FOR ALTERNATE SUPER-BLOCK FAILED. YOU MUST USE THE
-b OPTION TO FSCK TO SPECIFY THE LOCATION OF AN ALTERNATE
SUPER-BLOCK TO SUPPLY NEEDED INFORMATION; SEE fsck(8).

2.2 - repair by adding the r option

This can take a while, it took 60min for my 1,5TB Disk.

[script]
MyMac:~ jb$ fsck_hfs -fy /dev/disk1s2
** /dev/rdisk1s2
   Executing fsck_hfs (version diskdev_cmds-491.3~2).
** Checking Journaled HFS Plus volume.
** Checking extents overflow file.
** Checking catalog file.
   Invalid node structure
(4, 242826)
** The volume Grozor could not be verified completely.
iMac-de-la-familia:~ jb$ fsck_hfs -ypr /dev/disk1s2
   Executing fsck_hfs (version diskdev_cmds-491.3~2).
** Checking Journaled HFS Plus volume.
** Checking extents overflow file.
** Checking catalog file.
** Rebuilding catalog B-tree.
   Invalid node structure
(4, 242826)
   Invalid record count
(4, 242827)
   Invalid record count
(4, 242828)
   Invalid record count
(4, 242829)
   Invalid record count
(4, 242830)
   Invalid record count
(4, 242831)
** Rechecking volume.
** Checking Journaled HFS Plus volume.
** Checking extents overflow file.
** Checking catalog file.
   Missing thread record (id = 47443244)
   Incorrect number of thread records
(4, 21586)
   Incorrect number of thread records
(4, 21586)
** Checking multi-linked files.
   Incorrect number of file hard links
** Checking catalog hierarchy.
   Invalid directory item count
   (It should be 428 instead of 440)
   Missing thread record (id = 47443242)
   Invalid directory item count
   (It should be 0 instead of 1)
   Invalid directory item count
   (It should be 107 instead of 440)
   Invalid directory item count
   (It should be 673621 instead of 673646)
   Invalid volume file count
   (It should be 4308261 instead of 4309339)
** Checking extended attributes file.
   Incorrect number of extended attributes
   (It should be 1823295 instead of 1823257)
   Incorrect number of Access Control Lists
   (It should be 1823272 instead of 1823234)
** Checking multi-linked directories.
   Incorrect number of directory hard links
** Checking volume bitmap.
   Volume bitmap needs minor repair for orphaned blocks
** Checking volume information.
   Invalid volume free block count
   (It should be 19453978 instead of 7705538)
   Volume header needs minor repair
(2, 0)
** Repairing volume.
...

If your drive is sill not repaired :

[script]
   Invalid first link in hard link chain (id = 47694526)
   (It should be 49482808 instead of 49484125)
** Look for missing items in lost+found directory.
** The volume Grozor could not be repaired.

Then you can try to fix it again with your Disk utilities (Application/utilities)

Then if you are as lucky than I was, you will be able to mount under OSX. Now it is wise to backup important files (if not already done) and run a physical test.

Installer Debian Lenny x64 en RAID Logiciel

Jean-Baptiste HEREN — Sun, 30 May 2010 13:33:00 +0200

L'installation sur des disques configurés en RAID1 offre les avantages suivants :

Les données sont intégralement reproduites sur deux disques (miroir)
Cela permet en cas d'avarie dur un disque, de conserver un système fonctionnel
Il est aisé de remplacer un disque défectueux

Installation d'une debian 5.04 en RAID.

Après avoir récupéré une image disque depuis http://www.debian.org, préparé la machine avec au moins deux disques de capacité équivalente, on démarre.

L'installateur nous demande de sélectionner une langue, une zone géographique, un nom d'hôte.

1- Préparation des disques pour le RAID

Nous allons créer trois partitions gérées en RAID :

/boot : contenant le noyau, démarré par le gestionnaire de boot GRUB
/ : Le root contient l'ensemble des données système
swap

Il faudra avoir exactement les mêmes tailles de partitions sur les deux disques.

Pour le partitionnement des disques, nous choisirons donc l'option "Manuel".

Nous allons créer deux partitions sur chaque disque :

la première pour le boot, 512M
le Root : 10G

Voici quelques copies d'écran illustrant les étapes de cette opération.

Voici ce que nous devrions obtenir à la fin de cette étape :

2- Configuration dur RAID Logiciel

Nous allons ici créer deux périphériques multidisque. Il s'agit de regrouper chacune de nos partitions identiques sur nos disques afin de créer un disque RAID.

Ces périphériques pourront être utilisés soit directement pour créer des partitions, soit pour monter des volumes physiques LVM.

Voici quelques images illustrant les différentes étapes :

Et voici le résultat attendu à cette étape:

Nous avons Maintenant deux périphériques RAID, exploitables comme s'il s'agissait de disques physiques, et sur lesquels nous pouvons créer nos partitions.

3- Création des partitions

Il ne nous reste donc plus qu'à attribuer nos périphériques RAID à un usage habituel (ext3 ou swap), de préciser les points de montage, puis de poursuivre notre installation de manière classique !

Sources :

http://dev.jerryweb.org/raid-fr/

http://www.supinfo-projects.com/fr/2005/lvm%5Fraid%5Flinux/2/

Build & use mod_auth_user_dir on debian 5.0 Lenny

Jean-Baptiste HEREN — Fri, 02 Apr 2010 12:34:00 +0200

In the need of a WebDAV functionnality that could allow one to share users personal directories with automated authentification, i found an apache module, auth_user_dir, that looked fine to me on http://www.genos.org :

You can create private user folders using the WebDAV protocol. Users will have only permissions to access to their own directory. Without this module, a htaccess file must be created inside every user folder. This is completely unmanageable when the number of users is big.

We can build a file server accessible from Windows and Linux using the WebDAV protocol and a web server like Apache. Because the protocol is build upon HTTP, the server uses ports TCP 80 (HTTP) and TCP 443 (HTTPS). These ports are commonly open in firewalls without any other special requirement.

The module appears to have been developed in 2004 an attached tutorial was referring to some patches that no longer exist. So here is explained the way I managed to build this module on debian in 2010. I also found some tricks to make the mysql authentication work. This is a little crap but it worked !

Prerequisites

first install the needed dev packages if those are not already on your system

# apt-get install apache2 \

apache2-threaded-dev \

libtool \

libltdl3 \

libltdl3-dev \

automake \

make \

libapache2-mod-auth-mysql \

libapache2-mod-ldap-userdir \

libapr1 \

libapr1-dev \

Get the module source code (or take the ready to make attached file)

# cd /root

# mkdir src

# cd src

# wget http://www.genos.org/downloads/mod_authuserdir-1.0.tgz

# tar -xzvf mod_authuserdir-1.0.tgz

# cd mod_auth_user_dir/

Then download the replacement files attached or patch

# patch -p1 < Makefile.am.diff

# patch -p1 < configure.in.diff

build the module

# aclocal

# autoconf

# automake -a

# ./configure --with-apache=/usr/include/apache2

Install the module

Build and install library

# apxs2 -i -a -n auth_user_dir libmod_auth_user_dir.la

Create the needed file for activation of this module in apache2, the debian way

# echo 'LoadModule mod_auth_user_module /usr/lib/apache2/modules/libmod_auth_user_dir.so' \

> /etc/apache2/mods-available/auth_user_dir.load

Enable the needed modules

# a2enmod auth_mysql

# a2enmod dav

# a2enmod dav_fs

# a2enmod auth_user_dir

Create database and tables for mysql_auth . (Here we assume you already have mysql running)

# mysqladmin -uroot -p create davusers

# mysql -uroot -p davusers

mysql> CREATE TABLE users (

> user_name CHAR(30) NOT NULL,

> user_passwd CHAR(20) NOT NULL,

> user_group CHAR(10),

> PRIMARY KEY (user_name)

> );

Add one user to database

> INSERT INTO `davusers`.`user_info` (

> `user_name` ,

> `user_passwd` ,

> `user_group`

> ) VALUES (

> 'test', 'test', ''

> );

Create folder for DAV & users subfolders

# mkdir /var/davshare

# mkdir /var/davshare/test

Change apache virtualhost configuration

Add the following directory to your virtualhost conf file

Alias /webdav /var/davshare

# Activate DAV module on this location

DAV On

# Specific auth_user_Dir Parameter

AuthUserDirRoot "/var/davsharev"

# auth_mysql parameters

AuthName "Secured access"

AuthGroupFile /dev/null

AuthUserFile /dev/null

AuthType Basic

AuthBasicAuthoritative Off

AuthMySQL On

AuthMySQL_Host localhost

AuthMySQL_User admin

AuthMySQL_Password mmypassword

AuthMySQL_Authoritative on

AuthMySQL_DB davusers

AuthMySQL_Password_Table user_info

AuthMySQL_Username_Field user_name

AuthMySQL_Password_Field user_passwd

AuthMySQL_Group_Field user_group

AuthMySQL_Group_Table user_info

AuthMySQL_Encryption_Types Plaintext PHP_MD5 Crypt_DES

require valid-user

</Limit>

</Location>

Restart Apache2

# /etc/init.d/apache2 restart

Does it works ?

If you are on a linux, then you can test your connexion with a command line client : cadaver.

# apt-get install cadaver

# cadaver my.virtual.host/webdav/test

cadaver should ask you a username and password. type the ones you added in mysql table and you should see a dav prompt like it :

dav:/webdav/> ls

Listing collection `/webdav/': succeeded.

Coll: test 0 Apr 2 16:14

Coll: notme 0 Apr 2 16:14

Then if you try to access to a folder different from the user you specified at logon, you will be asked for the correct username and password.

So it should work. the last issue is that users can see the list on folders in the dav share, even if they cannot see the contents. If anyone has an idea to fix this issue ...

Also here is a nice description on how you can get connected to a webDAV Share using Any operating system : http://plone.org/documentation/kb/webdav.

I hope this will help someone, it already helped myself :)