Spotlight

From Netatalk Wiki
(Difference between revisions)
Jump to: navigation, search
(Created page with "== Overview == Starting with version 3.1.0 Netatalk includes support for Apple's Spotlight technology for searching AFP volumes. == Requirements == Netatalk uses [https://live.g...")
 
Line 3: Line 3:
  
 
== Requirements ==
 
== Requirements ==
Netatalk uses [https://live.gnome.org/Tracker/ Tracker] as the metadata backend. Most Linux distributions and *BSD systems will provide the libtracker-sparql library which is available since Tracker version 0.7. This version is referred to as Tracker SPARQL from hereon in this document.
+
Netatalk uses [https://live.gnome.org/Tracker/ Tracker] as the metadata backend. Recent Linux distributions will provide the libtracker-sparql library which is available since Tracker version 0.7. This version is referred to as Tracker SPARQL from hereon in this document.
Other system like Solaris and systems derived from Solaris will only ship Tracker version 0.6 which only offers a much more restrcited feature set. We refer to this version and its featureset by Tracker RDF.
+
Other system like FreeBSD, Solaris and systems derived from Solaris will only ship Tracker version 0.6 which only offers a much more restrcited feature set. We refer to this version and its featureset by Tracker RDF.
 +
 
 +
Solaris users are advised to install Tracker from OpenCSW as this is at least version 0.15 and thus supports SPARQL.
  
 
Both Spotlight implementations, the one using Tracker SPARQL and the one using Tracker RDF, share some of the configuration options and features, where configuration and features differ we provide specifc information below.
 
Both Spotlight implementations, the one using Tracker SPARQL and the one using Tracker RDF, share some of the configuration options and features, where configuration and features differ we provide specifc information below.

Revision as of 15:53, 27 May 2013

Contents

Overview

Starting with version 3.1.0 Netatalk includes support for Apple's Spotlight technology for searching AFP volumes.

Requirements

Netatalk uses Tracker as the metadata backend. Recent Linux distributions will provide the libtracker-sparql library which is available since Tracker version 0.7. This version is referred to as Tracker SPARQL from hereon in this document. Other system like FreeBSD, Solaris and systems derived from Solaris will only ship Tracker version 0.6 which only offers a much more restrcited feature set. We refer to this version and its featureset by Tracker RDF.

Solaris users are advised to install Tracker from OpenCSW as this is at least version 0.15 and thus supports SPARQL.

Both Spotlight implementations, the one using Tracker SPARQL and the one using Tracker RDF, share some of the configuration options and features, where configuration and features differ we provide specifc information below.

Installation

The tracker packages are found via pkg-config, you may have to pass the version suffix as you may have a new version installed then the default 0.12, eg

 $ pkg-config --list-all | grep tracker
 tracker-extract-0.16  tracker-extract - Tracker : A library to develop metadata extractors for 3rd party file types.
 tracker-sparql-0.16   tracker-sparql - Tracker : A library to perform SPARQL queries and updates in the              Tracker Store
 tracker-miner-0.16    tracker-miner - A library to develop tracker data miners

So:

 --with-tracker-pkgconfig-version=0.16

If you're using Solaris and Tracker from OpenCSW, then you need to set the PKG_CONFIG_PATH environment variable and add the --with-tracker-prefix configure option:

 PKG_CONFIG_PATH=/opt/csw/lib/pkgconfig ./configure ... --with-tracker-prefix=/opt/csw ... --with-tracker-pkgconfig-version=0.16

You can check the configure output:

 checking for TRACKER... yes
 checking for TRACKER_MINER... yes
 ...
 Configure summary:
 ...
   AFP:
     Spotlight: yes
 ...

Configuration

Enable Spotlight in $sysconfdir/afp.conf:

 [Global]
 ...
 spotlight = yes
 ...

Tracker SPARQL

All Netatalk volumes will automatically be indexed by Tracker and are then searchable from AFP clients.

Tracker RDF

Add all volume paths that should be searchable to $sysconfdir/tracker/tracker.cfg

 ...
 [Watches]
 # List of directory roots to index and watch (separator=;)
 WatchDirectoryRoots=/Volumes/test;/another/volume
 ...


For large volumes the indexing may take some time.

Limitations and notes

Large filesystems

Tracker on Linux uses the inotify Kernel filesystem change event API for tracking filesystem changes. On large filesystems this may be problematic since the inotify API doesn't offer recursive directory watches but instead requires that for every subdirectoy watches must be added individually.

Tracker RDF

The mapping of certain simple and of complex Spotlight to Tracker RDF queries is imperfect. Also, Tracker RDF filename searches are case sensitive! As a result there are two noticable restrictions:

  1. On a Mac, a query entered in the Spotlight search menu or in a Finder search toolbar is meant to express "search any metadata field, file name or content for this string". With Netatalk and Tracker RDF only filenames will be searched.
  2. Searching files content must be done through explicitly adding a '"Contents" contains ...' filter.

Toolbar search is name search.png Searching for content.png

Supported metadata attributes

Tracker SPARQL

The following list is the complete set of supported metadata attributes in search queries

Description Spotlight Key
Name kMDItemDisplayName, kMDItemFSName
Document content (full text search) kMDItemTextContent
File type _kMDItemGroupId, kMDItemContentTypeTree
File modification date kMDItemFSContentChangeDate, kMDItemContentModificationDate, kMDItemAttributeChangeDate
Content Creation date kMDItemContentCreationDate
The author, or authors, of the contents of the file kMDItemAuthors, kMDItemCreator
The name of the country where the item was created kMDItemCountry
Duration kMDItemDurationSeconds
Number of pages kMDItemNumberOfPages
Document title kMDItemTitle
The width, in pixels, of the contents. For example, the image width or the video frame width kMDItemPixelWidth
The height, in pixels, of the contents. For example, the image height or the video frame height kMDItemPixelHeight
The color space model used by the document contents kMDItemColorSpace
The number of bits per sample kMDItemBitsPerSample
Focal length of the lens, in millimeters kMDItemFocalLength
ISO speed kMDItemISOSpeed
Orientation of the document. Possible values are 0 (landscape) and 1 (portrait) kMDItemOrientation
Resolution width, in DPI kMDItemResolutionWidthDPI
Resolution height, in DPI kMDItemResolutionHeightDPI
Exposure time, in seconds kMDItemExposureTimeSeconds
The composer of the music contained in the audio file kMDItemComposer
The musical genre of the song or composition kMDItemMusicalGenre

Tracker RDF

The following list is the complete set of supported metadata attributes in search queries

Description Spotlight Key
Name kMDItemDisplayName, kMDItemFSName
Document content (full text search) kMDItemTextContent
File type _kMDItemGroupId, kMDItemContentTypeTree
File modification date kMDItemFSContentChangeDate, kMDItemContentModificationDate, kMDItemAttributeChangeDate
Content Creation date kMDItemContentCreationDate
The author, or authors, of the contents of the file kMDItemAuthors, kMDItemCreator
The name of the country where the item was created kMDItemCountry
Duration kMDItemDurationSeconds
Number of pages kMDItemNumberOfPages
Document title kMDItemTitle
The width, in pixels, of the contents. For example, the image width or the video frame width kMDItemPixelWidth
The height, in pixels, of the contents. For example, the image height or the video frame height kMDItemPixelHeight
Focal length of the lens, in millimeters kMDItemFocalLength
ISO speed kMDItemISOSpeed
Orientation of the document. Possible values are 0 (landscape) and 1 (portrait) kMDItemOrientation
Exposure time, in seconds kMDItemExposureTimeSeconds
The musical genre of the song or composition kMDItemMusicalGenre

Using Tracker commandline tools on the server

First, Netatalk must be running. Then setup relevant environment variables, adjust the prefix /usr/local/netatalk to match your configured paths:

 $ su
 ****
 # cat trackerenv.sh 
 export DBUS_SESSION_BUS_ADDRESS="unix:path=/tmp/spotlight.ipc"
 export XDG_DATA_HOME=/usr/local/netatalk/var/netatalk
 export XDG_CACHE_HOME=/usr/local/netatalk/var/netatalk
 export XDG_CONFIG_HOME=/usr/local/netatalk/etc
 # . trackerenv.sh
 #

Tracker SPARQL

 # tracker-search QUERY
 ...
 # tracker-info PATH
 ...

Tracker RDF

 # export DBUS_SESSION_BUS_ADDRESS=unix:path=/tmp/spotlight.ipc
 # cat file.rdf
 <rdfq:Condition>
     <rdfq:and>
         <rdfq:contains>
             <rdfq:Property name="File:Name" />
             <rdf:String>SEARCHSTRING</rdf:String>
         </rdfq:contains>
     </rdfq:and>
 </rdfq:Condition>
 # tracker-query -p file.rdf File:Name
 ...
 # tracker-info -m File:Mime PATH
 ...

Important: you MUST specify an -m option with tracker-info, otherwise trackerd might crash probably due to a bug in Tracker.

References

  1. MDItem
  2. Tracker

Changelog

Personal tools
Namespaces
Variants
Actions
Navigation
Toolbox