Don't Crash When Reading Metadata

Each time a tooltip is shown in Dolphin or the Information Panel is enabled the metadata of a file must be retrieved:

History

When Dolphin has been written for KDE SC 3 this was straight forward: The class KFileMetaInfo has been used and the metadata was read in the context of the application within a short period of time. The fast retrieving was possible by using the hint KFileMetaInfo::Fastest that assured only up to 64 KByte of the file are read. This limit was sufficient for getting the most important metadata.

With the transition to KDE SC 4.0 KFileMetaInfo internally has been rewritten to use Strigi for getting the metadata. Like Dolphin and other components in KDE SC 4.0 also KFileMetaInfo and some metadata analyzers of Strigi had rough edges:

• The KFileMetaInfo::Fastest hint simply has been ignored. Hovering a big video file resulted in reading the whole file. As this reading has been done in the context of the application Dolphin was completely blocked during that period of time.
• When implementing the KFileMetaInfo::Fastest hint later it turned out that a few analyzers of Strigi simply decided to ignore this limit.
• To bypass a blocking Dolphin I've moved the reading of the metadata into a thread. This worked well until it turned out that some analyzers are not reentrant and yet-another-workaround was required to prevent two threads using a similar analyzer.
• Sebastian Trüg (although not responsible for this part of the code) investigated some cool work into KFileMetaInfo and found a way to let all Strigi analyzers respect the 64 KByte limit without changing the analyzers. I was very happy about this until it turned out that some analyzers crash when they get streams ending after 64 KByte...

Current State

The following things work now in KDE SC 4.6:

• The quality of a lot of Strigi analyzers has been improved during the last years and most of them work very reliable.
• The analyzers work fast by getting only 64 KByte to read.
• In comparison to KDE SC 3 the metadata is read in a custom thread which makes the user interface even more responsive.

Still open:

• Some analyzers are not reentrant. This is not a big showstopper as reading metadata in parallel is anyhow quite rare so the workaround to serialize the access won't result in a noticable slowdown from a users point of view.
• Some analyzers crash (and hence also Dolphin crashes).

From a users point of view it is not important whether Dolphin crashes because of an analyzer or because of an issue in the Dolphin code. Although most of the analyzers are very stable in the meantime it is still a pain for users that e.g. have some Powerpoint-files where the corresponding analyzer simply cannot scope with the 64 KByte limit.

KDE SC 4.7

I understand that writing metadata analyzers can be very tricky, but I was always convinced that not crashing when reading data is not difficult. But no matter whether I'm right or wrong with this assumption: Fact is that more than 3 years after the release of KDE SC 4.0 a few analyzers still crash and seem to be without maintainers that care about this.

Well, with KDE SC 4.7 those crashes will not result in crashing Dolphin anymore. I've moved the reading of the metadata into a custom process. If an analyzer crashes then the worst thing that happens is that no metadata is retrieved. The nice thing is that this also solves crashes for other components that read metadata: E.g. the fileproperties-dialog or overwrite-dialog of kdelibs. In retrospective I should have done this already a lot earlier, but the fact that the old approach worked so well in KDE SC 3 just made me resist to use a dedicated process for this. In the meantime I'm convinced that this approach is the right way in the longterm: It is simply risky if the stability of an application or some kdelibs-dialogs depend from external plugins where the quality of the code varies a lot.

So let's hope this is my last metadata-related post and that this topic is solved finally :-)

Update: Carsten Pfeiffer pointed out that in KDE 3 the KFileMetaInfo::Fastest hint did not restrict the parsing to 64 KByte. Instead it was up to the analyzers itself how they provide metadata that can be calculated cheaply. Too sad that most of the current Strigi analyzers don't support this.