Category Archives: Linux

KUbuntu 18.10 update : strange X11 problem, complex solution

Category : Linux

Upgrade, then reboot…

After the upgrade from 18.04 to 18.10, SDDM failed to start. This happened with kernel 4.18.0-10 and 4.15.0-36 (it was the chance to discover grub-reboot). The machine is a virtual machine, and PCI devices are using virtio-pci driver for example.

I looked into /var/log/Xorg.0.log and found many error messages, the first one being :

(EE) open /dev/dri/card0: No such file or directory
See Xorg log file and crash file attached.

Then I looked at the output of lspci -vvnn and it seems that no kernel driver manages the VGA Controller. According to the following message, the missing kernel driver may be : bochs-drm or virtio-vga
https://www.redhat.com/archives/libvir-list/2015-August/msg00220.html

Asking for help

As usual when something goes wrong and it seems it’s not my fault, I’ve asked for some help. I opened a bug on Launchpad :
https://bugs.launchpad.net/ubuntu/+source/xorg-server/+bug/1802950

Unfortunately that hasn’t been very helpful, mostly because I have never been able to upload a crash report. The standard crash report tool works under X11, which was of course not available. The command line tools didn’t work for me but I didn’t find why. So I left it sleep for some time

A new hope

There was a hint in the developer’s comments that there was some hope to make it work again in the future: Ubuntu could include the missing module in the kernel they provide. It was removed in 2014, and could be back as there is bug 1795857 from October 2018 asking for it. But such sidelines problem doesn’t get much attention, so it’s not included as of January 2019.

Go to the source

Then I thought that it’s only a module, and I could build it myself from the sources of the kernel. I didn’t know if it was easy under Ubuntu, but I already build kernels on Gentoo for 15 years, so I wanted to give it a try.

First I installed the kernel sources package:

 apt-get install linux-source-4.18.0

When I looked at its content I was disappointed, because it’s only a tarball, but it’s easy enough to decompress:

cd /usr/src/linux-source-4.18.0/
tar xjf linux-source-4.18.0.tar.bz2
cd linux-source-4.18.0/

I tried the usual make oldconfig but some packages are necessary to make it work:

apt-get install bison
apt-get install flex
apt-get install libelf-dev

From this point, I could go forward with my idea:

make oldconfig
make menuconfig

In the menus I searched and enabled the module CONFIG_DRM_BOCHS=m
Then I launched the compilation:

make modules

Now you have time to take a break and get coffee, because Ubuntu enables many modules that are compiled by this command. It also takes space: the whole kernel tree with all compilation results and intermediate files amounts to 15 Gb !

At the end of this compilation, I installed manually the only module I’m interested in, because I don’t want to cripple the modules installed from standard packages:

mkdir /lib/modules/4.18.0-12-generic/kernel/drivers/gpu/drm/bochs
cp drivers/gpu/drm/bochs/*.ko /lib/modules/4.18.0-12-generic/kernel/drivers/gpu/drm/bochs
depmod -a

Now is the moment of truth, I try to load this module in the running kernel:

modprobe bochs-drm

It didn’t work, but it was not obvious because modprobe didn’t show any message, so I searched elsewhere and found something in dmesg output:

bochs_drm: version magic '4.18.17 SMP mod_unload ' should be '4.18.0-12-generic SMP mod_unload '

I didn’t know where this was defined nor if I could do something about it. If it’s not the same version, maybe I’ve the wrong kernel sources !

However I found it in the main Makefile of the kernel so I decided to make a further attempt by changing the values in that file, even though I’ve no idea of the potential consequences… So I changed the values:

SUBLEVEL = 17
EXTRAVERSION =

To the values expected by the running kernel:

SUBLEVEL = 0
EXTRAVERSION = -12-generic

Then I restarted a clean compilation:

make clean
make modules
# Get more coffee
cp drivers/gpu/drm/bochs/*.ko /lib/modules/4.18.0-12-generic/kernel/drivers/gpu/drm/bochs
depmod -a
modprobe bochs-drm
systemctl start sddm

And then I got SDDM back on screen !

I hope bochs-drm will be included in future standard kernels, as asked for in bug 1795857, because until it is included, these steps will be necessary after each update of the standard kernel.

Conclusion

That was not a simple solution to my problem. Did it work ? Yes ! Do I recommend it to anyone ? No, this method is too brittle and takes too much time.

But it shows how Open Source makes it possible to solve your problems without the help of the author or packager of the software you want to use, and even do things that where not anticipated by them.

Sorting files by date

Category : Bash Linux

My phone saves pictures with names in the format

IMG_YYYYMMDD_HHMMSS.jpg

I upload all images to my machine in a single directory, but I want to sort them by date. I use the following commands to sort them, in two phases :

First, create the directories

ls|cut -d_ -f2|sort -u|while read date; do
mkdir -p /data/pictures/${date:0:4}/${date:0:4}-${date:4:2}-${date:6:2};
done
  • ls gives all image names
  • cut -d_ -f2 extracts the date of each picture
  • sort -u removes duplicate dates with the -u (unique) option
  • while read date loops on all dates to run the next command
  • mkdir -p creates the directory for the date, and eventually the year
  • The syntax ${date:0:4} extract parts of the date. In this case the year

Then move images into directories

ls|cut -d_ -f2|sort -u|while read date; do
mv *$date* /data/pictures/${date:0:4}/${date:0:4}-${date:4:2}-${date:6:2};
done

Only the command inside the loop is different :

  • mv *$date* moves all pictures of the same day in the target directory

If the date is not in the file name, but is available in the directory, you can get it with the command ls --full-time and parse the output, or you can use the command stat to get exactly what you want, as it gives you full control on the output.

Converting a Python project to PEP8 with Shell

Category : Linux Python

When I started Python I didn’t learn PEP8 immediately so my code was not following the standard format. As I know Bash quite well, I’ve converted my projects later, with the following commands.

Convert tabs to 4 spaces

find . -name '*.py'|xargs pep8|grep W191|cut -d : -f1|xargs sed -i "s/\t/    /g"

Suppress spaces when they’re alone on the line

find . -name '*.py'|xargs pep8|grep W293|cut -d : -f1|sort -u|xargs sed -i "s/^ *$//"

Suppress spaces at end of line

find . -name '*.py'|xargs pep8|grep W291|cut -d : -f1|sort -u|xargs sed -i "s/ *$//"

Add spaces after “:”, “,”

find . -name '*.py'|xargs pep8|grep E231|cut -d : -f1|sort -u|xargs sed -i 's/:\([^ ]\)/: \1/g'
find . -name '*.py'|xargs pep8|grep E231|cut -d : -f1|sort -u|xargs sed -i 's/,\([^ ]\)/, \1/g'

Suppress spaces before or after {} [] ()

find . -name '*.py'|xargs pep8|grep E201|cut -d : -f1|sort -u|xargs sed -i 's/{ */{/g'
find . -name '*.py'|xargs pep8|grep E201|cut -d : -f1|sort -u|xargs sed -i 's/\[ */[/g'
find . -name '*.py'|xargs pep8|grep E201|cut -d : -f1|sort -u|xargs sed -i 's/( */(/g'
find . -name '*.py'|xargs pep8|grep E202|cut -d : -f1|sort -u|xargs sed -i 's/ *}/}/g'
find . -name '*.py'|xargs pep8|grep E202|cut -d : -f1|sort -u|xargs sed -i 's/ *]/]/g'
find . -name '*.py'|xargs pep8|grep E202|cut -d : -f1|sort -u|xargs sed -i 's/ *)/)/g'

Suppress spaces before “(“, “:”

find . -name '*.py'|xargs pep8|grep E211|cut -d : -f1|sort -u|xargs sed -i 's/ *(/(/g'
find . -name '*.py'|xargs pep8|grep E203|cut -d : -f1|sort -u|xargs sed -i 's/ *:/:/g'

Add a blank line at end of file

find . -name '*.py'|xargs pep8|grep W292|cut -d: -f1|while read f; do echo >> $f; done

Add a blank line before “def” or “class”, to be run two times

find . -name '*.py'|xargs pep8|grep E302|cut -d : -f1|sort -u|xargs sed -i 's/^def/\ndef/'
find . -name '*.py'|xargs pep8|grep E302|cut -d : -f1|sort -u|xargs sed -i 's/^class/\nclass/'

Sometimes I don’t really want to cut long lines, so I keep lines up to 120 characters

find . -name '*.py'|xargs pep8 --max-line-length=120

I think that 80 is to restrictive and based on terminals of the last century or on paper sizes that no one uses any more.

However, I’ve learned how to cut long lines with chained calls.

Before:

compte.dernier_solde = SoldeJour.objects.filter(compte_id=compte.id).order_by('-date_solde')[0].solde

After, with parenthesis to tell the Python parser that a single expression is on multiple lines:

compte.dernier_solde = (
SoldeJour
.objects
.filter(compte_id=compte.id)
.order_by('-date_solde')[0]
.solde
)

I’ve also configured Scite, my favorite editor, for spaces and tabs.

use.tabs=0
tabsize=4
indent.size=4

Search

Recent Posts

Categories

Meta

© Copyright 2020. All Rights Reserved