Montag, 26. Januar 2015

Migration to "Cubietruck" finished

Hi,

due to a new job since the beginning of the year and moving to a new country I was a bit stressed. In addition to this, my provider changed from IPv4 to IPv6 at a DSLite line.

So my port forwarding is no longer working and I had to organize all the stuff to get access to my mails again :-). Well, that's a disadvantage of my solution but I hope this will be solved somehow.

So my script to install Kolab on my truck:


export RELEASE=3.3
export RELEASEDIR=kolab_3.3_src
mkdir -p /mnt/kolab/$RELEASEDIR
mkdir -p /mnt/kolab/kolab_${RELEASE}_deb
echo "deb-src http://obs.kolabsys.com:82/Kolab:/$RELEASE/Debian_7.0/ ./" > /etc/apt/sources.list.d/kolab.list
echo "deb-src http://obs.kolabsys.com:82/Kolab:/$RELEASE:/Updates/Debian_7.0/ ./" >> /etc/apt/sources.list.d/kolab.list
#echo "deb http://kolab-deb.zion-control.org /" >> /etc/apt/sources.list.d/kolab.list
echo "Package: *" >  /etc/apt/preferences.d/kolab
echo "Pin: origin obs.kolabsys.com" >>  /etc/apt/preferences.d/kolab
echo "Pin-Priority: 501" >>  /etc/apt/preferences.d/kolab
wget -qO - http://obs.kolabsys.com:82/Kolab:/$RELEASE/Debian_7.0/Release.key | apt-key add -
wget -qO - http://obs.kolabsys.com:82/Kolab:/$RELEASE:/Updates/Debian_7.0/Release.key | apt-key add -
aptitude update
cd /mnt/kolab/$RELEASEDIR
echo Get debian sources

wget http://obs.kolabsys.com:82/Kolab:/$RELEASE/Debian_7.0/Packages -O packages.txt
wget http://obs.kolabsys.com:82/Kolab:/$RELEASE:/Updates/Debian_7.0/Packages -O packages_updates.txt
apt-get -ym source `grep Package packages.txt | awk '{print $2}' | grep -v "kolab-ucs" | sort -u`
apt-get -ym source `grep Package packages_updates.txt | awk '{print $2}' | grep -v "kolab-ucs" | sort -u`
apt-get -y build-dep `grep Package packages.txt | awk '{print $2}' | grep -v kolab-ucs | sort -u`
dpkg -i 389-ds-base-libs_1.2.11.29-0_armhf.deb
dpkg -i 389-ds-base-dev_1.2.11.29-0_armhf.deb
cd python-icalendar-3.4/
debuild -us -uc -b
cd ..
apt-get install python-dateutil python-tz
dpkg -i python-icalendar_3.4-1_all.deb
cd libcalendaring_4.9.0
debuild -us -uc -b
cd ..
dpkg -i libcalendaring_4.9.0-3_armhf.deb
dpkg -i libcalendaring-dev_4.9.0-3_armhf.deb
apt-get install libboost-program-options-dev
cd libkolabxml-1.1~dev20140624/
debuild -us -uc -b
cd ..
dpkg -i libkolabxml1_1.1~dev20140624-0~kolab1_armhf.deb
dpkg -i libkolabxml-dev_1.1~dev20140624-0~kolab1_armhf.deb
cd libkolab-0.6~dev20140624/
debuild -us -uc -b
cd ..
dpkg -i libkolab0_0.6~dev20140624-0~kolab1_armhf.deb
dpkg -i libkolab-dev_0.6~dev20140624-0~kolab1_armhf.deb
apt-get -y build-dep `grep Package packages.txt | awk '{print $2}' | grep -v kolab-ucs | sort -u`
apt-get -y build-dep `grep Package packages_updates.txt | awk '{print $2}' | grep -v kolab-ucs | sort -u`
ls -ld * | grep '^d' | awk '{print $9}' | while read verz
do
        cd $verz
        debuild -us -uc -b
        cd ..
done

cp *deb ../kolab_${RELEASE}_deb/
cd ../kolab_${RELEASE}_deb
dpkg-scanpackages -m .  | gzip -c9 > Packages.gz

After a while it was installed and I had to move all mails to my truck. I decided to reorganize all the mails according to the recipient in a separate folder (because of my C@tch@ll). I used "imapfilter" for that and for this I wrote a little LUA script:

function processMailbox(raspberryPi,cubietruck, mb,subfolders)
        -- Status
        total, recent,unseen,nextid = raspberryPi[mb]:check_status()
        -- Alle Nachrichten auswaehlen
        results = raspberryPi[mb]:select_all()
        number = 1
        for _, message in ipairs(results) do
                print ("")
                mailbox, uid = table.unpack(message)
                header = mailbox[uid]:fetch_header()
                recipient = string.gsub(header, ".*for <([A-Za-z0-9-]+@example.com)>.*","%1")
                if (recipient:find(":")) then
                        headerTo = mailbox[uid]:fetch_field('To')
                        if (headerTo == nil) then
                                recipient = "invalid_to@example.com"
                        else
                                headerTo = headerTo:gsub("\r","")
                                headerTo = headerTo:gsub("\n","")
                                headerTo = headerTo:gsub(" ","")
                                headerTo = headerTo:gsub("To: ","")
                                if (headerTo == "") then
                                        recipient = "invalid_to@example.com"
                                else
                                        recipient = headerTo:gsub("To: ","")
                                end
                        end
                end
                if (recipient:find("<")) then
                        recipient = recipient:gsub(".*<(.*)>","%1")
                end
                toFolder = string.gsub(recipient,"(.-)(@.*)","%1")
                messageId = mailbox[uid]:fetch_field("Message-id")
                messageId = string.gsub(messageId,".-<(.-)>","%1")
                subject   = mailbox[uid]:fetch_field('Subject')
                subject   = subject:gsub("\r"," ")
                subject   = subject:gsub("\n"," ")
                print('Processing : <' .. mb .. '>')
                print('Subject    : <' .. subject .. '>')
                print('Recipient  : <' .. recipient .. '>')
                print('Folder     : <' .. toFolder .. '>')
                print('ID         : <' .. messageId .. '>')
                prozent = number / total * 100
                print('Number     : <' .. number .. '/' .. total .. '> in ' .. mb .. ' (' .. prozent .. ' %)')
                cubietruck:create_mailbox('Archive/' .. toFolder)
                toMove = raspberryPi[mb]:contain_field("Message-id",messageId)
                for _, messageM in ipairs(toMove) do
                        tmMB,tmUID = table.unpack(message)
                end
                toMove:copy_messages(cubietruck['Archive/' .. toFolder])
                number = number + 1
        end
        -- process all subfolder
        if (subfolders) then
                rPi_Mailboxes, rPi_Folders = raspberryPi:list_all(mb)
                for _, mbSub in ipairs(rPi_Mailboxes) do
                        processMailbox(raspberryPi,cubietruck,mbSub,true)
                end
        end
end
---------------
--  Options  --
---------------
options.timeout = 120
options.subscribe = true
options.create = true
----------------
--  Accounts  --
----------------
-- Quelle ist Pi und Ziel ist Truck
raspberryPi = IMAP {
    server = 'raspberrypi.example.com',
    username = 'username@example.com',
    password = ':-)',
}
cubietruck = IMAP {
    server = 'cubietruck.example.com',
    username = 'username@example.com',
    password = ';-)',
}
processMailbox(raspberryPi,cubietruck,'Subfolder',false)


Don't hesitate to ask any questions

Greetz

Samstag, 3. Januar 2015

Repository updated for Raspberry Pi and migration to Cubietruck started

Hello,

currently my +Raspberry Pi compiles the packages to the latest version available. (Update from 01.01.2015)

In the next couple of weeks I'm going to migrate from +Raspberry Pi  to my new Cubietruck. I decided to upgrade to a new SBC because the Cubietruck has more power (2 GB Ram instead of 512MB, ARMv7 instead ARMv6 a.s.o.)

For the Cubietruck I decided to go the debootsrap way for a debian image and followed these instructions :
After that I had an amazing fast boot sequence on my "truck". :-) 
I had to fix an  issue on my host because the module "binfmt_misc" was not integrated in my kernel which I compiled myself.

The main difference beetween Raspberry Pi and the Cubietruck is the CPU.
Debian is available for ARMv7 CPUs as "armhf" and for ARMv6 CPUs as "armel" (soft float). The difference is the performance. If you use "armel" on the Raspberry Pi you will loose lots of performance, that's why the guys from Raspbian created their own repository for ARMv6 and "hard float". They compiled nearly every package for the Pi and I did that for the +Kolab packages.

The debian-armhf repository does not contain all of the packages which are necessary for the Kolab installation so my "truck" is compiling all the stuff as well. :-)

Greetz
Andreas