Profiling

Although I like Criminal Minds, this blog entry is less thrilling: it's about profiling applications like Dolphin.

When I faced a performance issue in Dolphin, I usually used valgrind in combination with callgrind to find the bottleneck (valgrind --tool=callgrind dolphin -nofork). The output can be visualized by KCacheGrind in a nice way:


One big benefit of Valgrind is that it gives 100 % coverage of user-space code. But as usual there are rarely benefits without drawbacks: As valgrind is a kind of virtual machine, that uses just-in-time compiling, the application runs around 5 times slower. This can get a problem if timers are involved in the application. Also performance issues in combination with threads are tricky to identify, because valgrind does not give a non-obtrusive performance overview of the overall system.

So I started to get familiar with OProfile during the last week. OProfile is a non-obtrusive system-wide profiler, which means that the application runs (nearly) at the same speed as without profiler. OProfile gives an overview about the workload of all processes in the system. This is very useful for Dolphin, as the overall performance of Dolphin depends also on e. g. Nepomuk and asynchronous operations to e. g. get file previews. It is possible to convert the OProfile output to a readable KCachegrind file by 'opreport -gdf | op2calltree'.

Dependent on the usecase, both tools are really helpful. The next thing on my TODO-list is to get more familiar to locate I/O bound bottlenecks: When reading the number of sub directories for 20000 directories, the bottleneck is definitely not on the CPU side (in this case neither Valgrind nor OProfile can detect the bottleneck). All in all I hope that I find the time during the KDE SC 4.5 cycle to improve the Dolphin performance in some areas with the help of these tools.

Visiting Openismus in Berlin

On February the 1st I started my new job at Openismus. Most of the time I can work at home in Austria, but last Wednesday I got the chance to meet the Openismus team at their office in Berlin. Beside my first visit of Openismus, it was also my first visit of Berlin. I was very impressed by both of them: the team and Berlin.

I got the chance to do some trials with Qt on Maemo and to get familiar with what is working well and which parts still need a lot of investigations. All developers at Openismus are Gnome people, being the only one using KDE is not always easy ;-) But maybe it helped that Dolphin does not look so much different in comparison to Nautilus - I even got a KDE-sticker from an Openismus colleague.

I'm learning a lot of new stuff currently and will have less time during the next weeks for Dolphin. However this will change after mastering the usual time that is needed to get really productive. In the end I hope that I can apply the knowledge I gathered through KDE development with Dolphin for Openismus and also that Dolphin will benefit from the new things I learn at Openismus.

Internal Cleanups

Warning: This might be a quite boring blog entry for non-developers - no fancy screenshots and no new features... ;-)

Beside taking care to keep Dolphin simple and efficient for users, it is also very important for me to do the same for the code: It should be simple and easy to maintain for developers.

Sometimes this works quite well, but it also happens that parts of the code need to be refactored. It is quite common in commercial companies that developers don't get the chance to refactor the code: The user does not recognize it in the first sight and there is definitely a risk of having regressions.

I'm convinced that in the long-term keeping the code base clean pays off and results in less bugs and easier maintenance. Although some features like improved searching or version control support have been added for Dolphin in KDE SC 4.4, I also did some internal cleanups.

For KDE SC 4.5 I want to completely concentrate on fine-tuning the code base of Dolphin, so that some long standing bugs can be fixed in a clean manner:

Information Panel

I totally underestimated the complexity of the Information Panel before KDE SC 4.0 had been released and was never happy with the code-base. I tried to fix things from release to release (which can also be seen visually when looking at the screenshots from the Dolphin-News). Now with KDE SC 4.4 the code-base is a lot better, although there is still some work until it can be moved to kdelibs to get shared with other applications (I blogged about it already in Information Panel and Tooltips). One issue that needs definitely to be solved for KDE SC 4.5 is a non-blocking fallback solution for showing meta information even when Nepomuk and Strigi indexing are turned off (bug 193592). Also fixing the missing meta information in the properties dialog (bug 190588) is a must.

Column View

The column view has been added shortly before the feature freeze for KDE SC 4.0. I fell into the trap to assume that showing several columns cannot be much more difficult in comparison to the other views. The result was that the column view added a lot of complexity to the code and fixing column view related bugs was tricky. I cleaned up the code base for KDE SC 4.4 and am quite happy now: A lot of interfaces and workarounds could be removed, less code is needed and fixing column view bugs is straight forward. The beta releases and release candidates popped up some regressions, but they could be fixed quite fast.

Location bar ("breadcrumb")

Long before KDE SC 4.0 had been released, the location bar was only meant as prototype. It happened too fast, that we moved it to kdelibs to make it available for the file dialog... It was my fault not taking care to cleanup up the public interface of this widget before KDE SC 4.0 (honestly speaking: There really where more serious problems to solve to that time ;-)). I could not manage to do a cleanup for KDE SC 4.4, but a refactored version is already available on trunk. There are two nice "side effects" of this cleanup:

1. Dolphin can now easily remember the expansion state of detail-view trees when going back in history (I've to thank Frank Reininghaus a lot for his contributions in this area).
2. The Dolphin code could be simplified, as it is very easy now to remember history states.

There are also a lot of minor things I wanted to fix since KDE SC 4.0 already. For example in the icons view the wrapping of file names is only done nice on spaces or the minus symbol. Dots and underscores are handled like letters, which might lead to nasty wrappings like:
my_document.tx
t
instead of
my_document.
txt

The searching for Dolphin has already improved in KDE SC 4.4, but still needs a lot of work...

Things like these are what I'd like to fix in Dolphin for KDE SC 4.5, so please don't expect huge, new features :-)

BTW: I'm very happy to announce that on February the 1st I'll start working fulltime at Openismus. Most probably I'll get the chance to work with Qt for Maemo 6 (Harmattan).

Version Control Support

Before a feature is added in Dolphin, it is checked whether the feature is mandatory for Dolphin’s target user group. If this is not the case, it does not mean that the feature cannot be added; first it must be clarified whether the feature might be non-intrusive and adds a value for users outside the primary target user group. Non-intrusive is mainly related to the user interface. A feature that adds a lot of clutter to the main menu, context menus or toolbar might harm the target user group. In this case the feature will not be added.

A good example of a feature that is non-intrusive is the embedded terminal in Dolphin. It only requires one entry inside a sub menu, but adds great value for a lot of people that are not in the scope of Dolphin’s target user group.

Another kind of non-intrusive feature will be available for Dolphin in KDE SC 4.4: version control support. In KDE 4.4 at least a plugin for Subversion is given:


The version state emblems and additional entries in the context menu will only be shown if a directory is under version control, so this feature fulfills the requirements for being non-intrusive. The plugin still has a lot of rough edges and has mainly been written for testing the plugin interface. Still I use it regularly and it fulfills most of my needs.

It is possible for developers to write other version control plugins for Git, Mercurial or whatever, just by following the instructions given in the plugin interface. In KDE SC 4.5 it is planned to embed the plugins as part of the kdevplatform package, but this has not been finally decided yet. The benefit to offer the plugins as part of kdevplatform will be, that the existing infrastructure for version control integration can be reused by the plugins.

Big thanks go to Nuno Pinheiro, who created icons for the version control states!

Searching

If I compare the number of files on my current hard disk with the number of files I had 10 years ago, I'm quite amazed. How to deal with this increased number of files is topic of a lot discussions. Tagging, searching, filtering, relations, virtual folders... - terms, where from a KDE-developer's point of view Nepomuk comes to mind, which provides already all the technical base. Still there is the task to integrate Nepomuk into applications, so that users out there are able to benefit from this great technology.

It's a small step only, but at least one step further: During the last week I've integrated Addam Kidder's GSoC 2009 work into Dolphin. I know that some people don't see searching and filtering as "the solution" to deal with the increased number of files and I agree. Nepomuk offers a lot more, still the searching and filtering is a basic feature, that I personally require quite often during my workflow. The current state of the code is far from finished, still the basics are there now. As soon as the search-bar at the top gets focused, the search configurations pop up below as shown here:


Searching for files having the tag "Tree" is very simple. Selecting "Equal to" and the corresponding tag in the combo-boxes beside "Tag:" has the following result:


One goal of the interface was to minimize the number of clicks that are required to specify a search query. That's the reason why per default at least three (hopefully) commonly used search criterions are visible. For sure it is possible to create very complex queries with more criterions too:


Still I think users will use such complex search queries quite rarely. I had some discussions with Sebastian Trüg and Alessandro Sivieri regarding the faceted panel, that exists as prototype currently. The discussion convinced me that a common workflow for users might be:

• Fill the search-bar with some keywords and trigger the searching.

• Only if the number of returned files is too high, start a filtering step by step on those files.

This might sound obvious, but has implications on the user interface:

1. The number of clicks to activate a filter must be minimized.
2. It should still be powerful: The kind of filters should not be limited.
3. Doing filtering step by step means that the user interface may not get hidden after triggering the first filter.

Especially the combination of point 1 and 2 have requires doing some compromises. Limiting the number of clicks requires more screen space. As there is only a limited screen space, the number of shown filters must be decreased. Finding a balanced solution is quite tricky.

I'm also unsure which data the breadcrumb should show: Visualizing a string like "nepomuksearch:/+lastModified<2009-11-13 +rating<8 +tag:Peter" to the user is something where I doubt Apple would do. Any suggestions would be welcome.

That's also one purpose to write this blog entry: Please let me know your daily problems when trying to search and filter files from your hard disk. I hope that I can integrate your input until KDE 4.4... Thanks!

Information Panel and Tooltips

Dolphin for KDE 4.4 will provide a refactored information panel and refactored tooltips. The goals of the refactoring have been:

• Show the same data in the information panel and the tooltips. Up to now the information panel did not provide any information about the permissions or the owner of a file. The tooltips have shown this kind of information already, but they did not provide informations like rating, comments or tags.

• Prevent a possible blocking of the user interface, even when a slow Nepomuk backend or a buggy Strigi analyzer is installed. Although in KDE 4.3 some improvements had been done already, there have been still problems in this regard especially for tooltips.

• Improve the layout of the information. The rating, comment and tags information are not handled in a special manner anymore.

• Simplify the interface for editing tags and comments.

Although there is still a lot of finetuning necessary until KDE 4.4, it is obvious already that the tooltips...


... show the same information as the information panel:


Originally it was planned to move the refactored widget to kdelibs, so that applications like Mailody, Gwenview and Okular can show the informations in a similar way. However there have been some usecases especially in combination with Gwenview, where the interface of widget was not sufficient yet. So we decided to leave the widget in Dolphin for KDE 4.4, but I'm confident that we find a good solution until KDE 4.5.

Nepomuk and Dolphin

Sebastian Trüg already blogged about the Nepomuk Coding Sprint that happened last weekend in Freiburg. It was great to meet all the guys in person and to share ideas face to face. Sebastian was a great host and I liked the general spirit of the meeting :-)

I just wanted to give a short summary from a Dolphin point of view. On the weekend the following things have been improved for KDE 4.3:

• The labels for Nepomuk meta information have been fixed.

• The usability of the comment widget in the Information Panel has been improved.

• Performance problems when renaming several hundreds of items have been fixed. Originally I thought this has been caused by Nepomuk, but it was a Dolphin internal issue.

• We could track down the root cause for freezes when a tooltip is shown for huge PDF files and MPEG files. Originally we wanted to discuss whether we want to use Nepomuk instead of Strigi to get the meta information, but it turned out that only two Strigi analyzers need to be fixed to respect a maximum size setting.

Well, I was not as productive as I wanted to be. The main cause was that my development environment on the Notebook was not setup properly and it took me some time to fix it (until this sprint I did not use my Notebook for development).

The more interesting thing is what we’ll try to improve for KDE 4.4:

• Better search integration. Although Dolphin for KDE 4.3 offers a search box, this is just a first start. We did some discussions and Dolphin for KDE 4.4 will offer a user interface element to configure the search query in an easy manner. Maybe this interface element can be used by other applications too if it works well.

• Fix usability issues of the tag rating widget. The tag rating widget in the Information Panel has some usability issues. Gwenview goes into the right direction I think and the goal is to have a common widget for KDE 4.4.

• Sebastian introduced his Formatting Rule System which might be very useful for the Information Panel. It requires a lot of work as it is still in draft, so it’s unclear whether we can integrate this to KDE 4.4 already.

Generally I was impressed how much potential Nepomuk offers. I think it will take quite some time, but I’m convinced that real innovative things can be done in future by using Nepomuk (especially in combination with PIM related applications).