Dash for iOS, Android, Windows or Linux

TL;DR: Dash-like apps for other platforms have been released. Check out Dash for iOS, Velocity for Windows, Zeal for Linux and LovelyDocs for Android.

I get asked a lot to bring Dash to other platforms. That won’t happen, because:

  • I’ve got a lot to add to Dash on macOS and I can’t focus on any other platform
  • I’m a complete novice when it comes to developing for any other platform, so I wouldn’t do a great job

Looking for devs to bring Dash to other platforms

I am actively looking for developers of other platforms (iOS, Android, Windows or Linux) that would like to work on a Dash-like app, as their own project and for their own profit.

Later edit: I have found iOS, Windows, Linux and Android devs and Dash-like apps for these platforms should be released sometime in 2014.

What you’ll make:

  • An API documentation browser app for your favourite platform
  • You can give it your own personal twist or base it on Dash as much as you want
  • Release it as commercial, free or keep it for personal use, I don’t care

What you get:

  • My help, as much as I can
  • Free access to all of Dash’s docsets to be used by your app
    • This includes docset updates and I’m also covering the hosting costs

What you won’t get:

  • Access to Dash’s source code

What I want in return:

  • A link to Dash, on your app’s presentation website and inside the app

Some notes:

  • Don’t start working on this without contacting me first
  • I’ll give exclusivity to Dash’s docsets to whoever looks most promising
  • This is not a weekend project
  • Making an awesome documentation browser takes time

In case you’re considering making this a commercial project:

  • You really should. Dash is my only source of income
  • I really think this is a huge opportunity for indie devs
  • I have no idea how much money you’ll make

Linux Man Pages in Dash

Dash works great as a man page browser, but I sometimes get requests to make extra docsets containing the man pages of various flavours of Linux.

I’ve decided not to pursue these requests, because:

  • Updates for these docsets would be a nightmare, as man pages change a lot, individually.
  • I’d have to choose which man pages to include and which not to. I’d never be able to guess which obscure man page a user might want.

The current Man Pages docset solves these issues by indexing the man pages that are actually on your Mac.

The workaround

You can copy the man pages from any Linux box to your Mac and Dash will index them as part of the regular Man Pages docset.

Step by step instructions:

  1. Log into your Linux box
  2. Run man -w to list the folders that contain man pages
  3. Copy these folders to your Mac
  4. Optional, but highly recommended: use a batch renamer to rename all of the man page files to have a common prefix. This will help you differentiate between the default macOS man pages and the Linux ones
  5. Move the man pages anywhere on your MANPATH, or any folder from man -w

That’s it!

A Sneak Peek at Dash’s Future

Disclaimer: This is the current plan for Dash. Plans can change. Please purchase Dash for what it is now and NOT for what it might become.

Tabs

Current status: almost done
Release estimate: soon

Tabs have been implemented and are currently in beta testing. Please join the beta and help test this feature!

Later edit: Tabs have been released with version 1.9.0 of Dash.

New icon

Current status: started
Release estimate: before the end of time

Reda Lemeden is working on a new icon for Dash. The icon is meant to highlight Dash’s two main features: documentation and speed.

Feedback is greatly appreciated! Here’s a preview:

Docset repositories

Current status: planned
Release estimate: 4-10 months

The general idea is for users to be able to search in Preferences > Downloads for their favorite packages and easily install docsets for them.

In the initial release, the following repositories will be supported:

Further repositories will be added incrementally: GoDoc, Haskell’s Hackage, node.js’s npm and Perl PODs.

Annotations

Current status: postponed
Release estimate: before the end of time

Users should be able to extend the documentation pages, publicly or privately, individually or as a team. For example, users might want to add annotations about common pitfalls while using a certain class or method.

Users will be able to create, edit and view annotations and vote for the most useful ones.

Some work has been done for annotations, but currently it has been postponed in order to work on other features with higher priority. I expect to start working on this again after “Docset Repositories” is implemented.

How it will work: a table will be shown on the right side of the documentation page. The table expands when you hover your mouse over it. Scrolling is synced between the documentation page and the annotations table.

That’s it for now

Any and all feedback is highly appreciated. Please join the beta and help test the tabs feature right now!

SQLite FTS contains and suffix matches

SQLite is used by Dash to search through docset indexes. Originally, Dash used LIKE queries which were fast enough, but became increasingly slower as more docsets were added.

SQLite FTS is amazingly fast, but allows only prefix (e.g. query*) matches by default. For Dash, I needed to persuade it to also perform contains matches (e.g. *query*) or suffix matches (e.g. *query).

How it works

It’s simple, for each term I want to be able to search, I store all of its suffixes.

First of all, the table structure:

CREATE VIRTUAL TABLE searchIndex USING FTS4(suffixes)

Add the term NSString:

INSERT INTO searchIndex(suffixes) VALUES("NSString SString String tring ring ing ng g")

Search using suffix queries:

SELECT * FROM searchIndex WHERE suffixes MATCH 'string';

Or contains queries:

SELECT * FROM searchIndex WHERE suffixes MATCH 'str*';

Downsides

The only downside I could find was that the database got too large. To avoid this, I compress the data into its actual term.

The compress and uncompress functions behave in this way:

compress("NSString SString String tring ring ing ng g")
-> NSString

uncompress("NSString")
-> NSString SString String tring ring ing ng g

This compression reduces the database size to what it would be if only the actual terms were added (without all the suffixes).

Speed results

Searching over 1,110,381 terms (in 102 docsets) using contains queries:

Search for "string" using LIKE:    3.22 seconds
Search for "string" using FTS:     0.18 seconds

Search for "s" using LIKE:         6.87 seconds
Search for "s" using FTS:          0.22 seconds

Alternatives

I chose SQLite FTS because I was already familiar with SQLite and I also needed to work around some Dash-specific edge cases (e.g. how symbols are treated).

Depending on your project, these may be suitable alternatives:

  1. PostgreSQL’s wildspeed module
  2. For macOS or iOS apps: Search Kit

A Poor Man’s CDN

Hosting large and often-downloaded files can be tricky, especially when you want users to have decent download speeds and 100% availability. This is the story of how Dash’s docsets are hosted.

First, some data:

  • At the time of writing, there are 102 docsets hosted
  • The total size of these docsets is 1.5 GB (while archived)
  • Bandwidth requirements are in the range of 5-7 TB / month
  • It would cost about $600 / month to host them in a “regular” CDN (e.g. Amazon CloudFront). In contrast, my hosting only costs $20 / month (thanks to 4 VPSs from DigitalOcean)

Hosting the docsets

Some docsets are somewhat large, so download speeds need to be decent. This is achieved by hosting the files in different data centers:

  • 2 mirrors in New York (for North America)
  • 1 mirror in San Francisco (for North America and Asia)
  • 1 mirror in Amsterdam (for Europe – or at least Western Europe)
  • Extra mirrors can be added in less than 2 minutes to account for spikes

South America, Eastern Europe, Africa and Australia are not directly covered, but should still have alright download speeds, as no one complained yet. More mirrors will be added whenever DigitalOcean opens more data centers.

Load balancing

Dash performs latency tests on all available mirrors by loading a small file. The mirrors are then prioritised based on latency. Whenever a mirror goes down, Dash notices and avoids it. Mirrors that have almost the same latency (±0.03s) are considered equal and are chosen randomly.

This setup results in 100% uptime and really cheap bandwidth costs. I highly recommend you consider a similar setup if you need to host large files for your app.

Hosting the docset feeds

The docset feeds are just small XML files which Dash polls to check for updates. These files are requested a lot, on each Dash launch and every 24 hours afterwards. As each docset has its own feed and most users have more than one docset installed, about 320k HTTP requests are made each day.

These requests are easily handled by a nginx web server on a 512 MB VPS in New York and are also mirrored on GitHub. I tried using Apache but it would sometimes use over 1 GB of RAM while hosting these files and would end up completely failing, while nginx serves requests faster and uses less than 40MB of RAM. I’ll talk about my experiences with nginx in a future post.

Whenever Dash needs to load a feed, it launches 2 threads which race to grab the feed (from kapeli.com or from GitHub), whichever thread finishes first wins and its results are used. Most of the time, the kapeli.com thread wins.

The chances of both kapeli.com and GitHub being unavailable are very very small, so this approach resulted in 100% uptime so far.

It’s Alive!

I finally found some time to start a blog. I’ll try to write as much as I can about as many topics as I can, but I’ll also try to keep all posts short and to the point, including this one.

Do you want me to write about something in particular (e.g. you’re curious about how I implemented something)? Let me know: contact me.