Transitioning from Papers 3 to Bookends: Part 2 - The How

After making the radical decision to uproot my reference management system, I decided to take the plunge with Bookends ($73.99 for MacOS+iOS version with an additional $9.99 per year Pro features on iOS). I now had the seemingly formidable task of moving all my PDFs and their attached references from Papers into Bookends. Spoiler: I managed to do this with relative ease.

Here are the steps I followed for my transition from Papers to Bookends (including the associated setbacks and successes):

0. Trying out Bookends

Before moving your entire database(s) to Bookends, I suggest you try out some simple functionality by dragging and dropping (or searching for) a PDF of a paper and check whether an accurate reference is retrieved. You also have the option of autocompleting an entry (Refs → Autocomplete Paper or Cmd+Shift+C). Try editing a reference manually and get familiar with the Bookends interface. I strongly recommend reading the official website’s FAQ, tutorials, support as well as the provided documentation.

1. “Import References from Papers”

Bookends has a handy built-in option to import references from Papers (or Sente or EndNote): File → Import References

Outcome: All the references move into Bookends but not the PDFs (or attachments, in Bookends’ parlance).

Potential Problems: If you receive a “no references exported from Papers” error, try restarting Bookends (while keeping Papers open).

2. Set up a preferred PDF naming structure

Explore writing your own reference and citation formats

Bookends can automatically rename an imported PDF based on the reference that it eventually retrieves. For example, Bookends can make s2-342439.pdf into Thirumalai-2011-Journal-Geol.pdf after crawling through the text. If you’d like to, you can customize this structure by first making (or choosing) a format in the manager: Biblio → Formats Manager.

Outcome: You will be able to choose your preferred naming structure in the preferences.

3. Optional: Set yourself up for iOS usage

Designate the format for renaming PDFs in Bookends and assign the attachments’ folder to the one in your iCloud drive

If you intend on using Bookends on iOS, make sure you go ahead and download the app now. Follow instructions here to import a PDF through a search engine. Ensure that sync has been enabled; now, a ‘Bookends’ folder will be created in your iCloud directory.

On the desktop version, open up preferences and in the drag-down list, choose iCloud Folder for iOS Sync as your attachments folder. The default, inside Documents, does not jive if you want to have an iCloud-synced iOS version, so you might as well follow the above steps even if you don’t want to use the iOS version (the free app supports this as well).

Outcome: You have a folder titled “Bookends” under iCloud Drive.

4. Find all your Papers' PDFs:

Find and consolidate all the PDFs stored in Papers’ virtual library.

Since you’ve already imported references from Papers, Bookends will be populated, but they will not have PDFs attached to them. If imported correctly, it will yield an error that the PDF (with a long, machine-readable string name) cannot be found. This problem stems from Papers’ opaque Virtual Library and file handling system. But this also poses an advantage: now we simply need to gather all the PDFs wherever Papers stores them (with intact names!) and paste them into the Bookends’ folder (inside the iCloud drive).

Go to your Papers’ virtual library/database location in Finder to start this procedure. In this overarching structure, search for .pdf and then ‘add’ a search where you filter by PDF kinds.

Outcome: You are able to select all the originally-named Papers PDFs under one search.

5. Move the PDFs to your Bookends Folder

Before you move anything, make sure that Bookends on your desktop is closed.

Depending on the number of PDFs you have, I would recommend moving in segments. For context, I copied ~1500 PDFs (of ~5000) from the Papers’ search folder and pasted into the Bookends folder (in iCloud Drive) first before doing the rest. The time for the overall transition including the wait for iCloud to upload all these files (make sure there are no small ‘uploading’ clouds visible in Finder!) was about ~20 mins.

Outcome: You have all your Papers’ PDFs in the Bookends Folder.

6. Allow Bookends to do its thing

Batch Edit: rename all your attachments

Open up Bookends, and if the attachment folder has been set to the one under the iCloud drive appropriately, all your references ought to have PDFs attached to them! Scroll through the references and make sure that PDFs are attached.

Now, since nobody likes those cruddy long, undecipherable names that Papers provides, let’s rename the PDFs based on our chosen format. Refs → Global Change → Rename Attachments. This took about ~35 minutes for me.

And - that’s it!

Outcome: All your Papers’ references are in Bookends, with PDFs intact, AND the PDFs are all inside the Bookends folder, named based on the structure you chose! Voila!

If you followed these steps, you ought to be able to open up Bookends on iOS and slowly wait for the iCloud sync to weave its magic.

Transitioning from Papers 3 to Bookends: Part 1 - The Why

The Problem

Support for the desktop version of Papers 3, my erstwhile reference management software of choice, was discontinued and sales ceased last November. I have been using some version of Papers on the Mac and iOS for over seven years now, and I have really enjoyed using it on both platforms. The Papers app on iOS was especially useful, with its selective Dropbox-sync and night-reading features. Over the years however, there were many growing annoyances. The lack of significant updates was frustrating, and even when updates were offered, they were largely unable to keep up with operating system advances. Ever since Papers was bought over by ReadCube, I have been worried about the future direction of the software as well as the long-term durability of my reference management system.

Why a reference manager?

Considering the aforementioned problem, a fleeting thought I had was to archive/delete all of my 5000+ PDFs, save a BibTeX file, and give in to the constant connectivity of the attention-economy era: download PDFs from their source whenever I needed it. However, unlike what Spotify does for music (for me), after some thought, I realized that this approach would become troublesome when I’m in the field (>month downtime with no internet, etc.) or traveling. Furthermore, retrieving some hard-to-get PDFs or scans of papers I already would’ve been challenging. I wanted to have access to my papers.

How about simply keeping PDFs somewhere on my iCloud with a fixed naming convention and separately update a BibTeX file with reference information taken from Scholar? I knew I didn’t have to start from scratch because Papers 3 could generate one giant text file with all my references in BibTeX format. This approach was also not entirely appealing. I knew that such a strategy could become unwieldy real quick for an ever-increasing number of papers, especially if I wanted to go inside and edit some references along the way (something that always happens). Also, it would’ve been painful to generate a revised .bib file with a sub-selection of citations for particular projects. Finally, if I ever wanted to use Word or another WYSIWYG editor, citation management would’ve become… a chore. Considering today’s technological umbrella, I don’t think asking for a half-decent reference management software is a tall ask.

What would my ideal reference manager look like?

  • Light, powerful, and not prone to crashing
  • Ability to attach PDFs to references
  • Transparent file handling and archival
  • Ability to handle a LOT of PDFs
  • Automatically “look” through a PDF and crawl the web for its full reference accurately
  • Ability to generate bibliographies or list of references and citations in any format I wanted (preferably customizable)
  • Have a PDF-editing interface where I can annotate or make notes on a paper
  • Ability to batch processes the references of several PDFs
  • Ability to have smart groups and smart search
  • Ability to slice and dice my papers in any way I’d like (e.g., view by journal/authors/keywords, etc.)
  • Syncs with cloud-service of choice (preferably iCloud so I can get out of the Dropbox ecosystem!)
  • Preferably has its own app on iOS via cloud interface
  • An affordable payment plan

Although Papers satisfied some of these constraints, as mentioned above, it had severe limitations, the most frustrating of which was its clunkiness and tendency to crash. Moreover, Papers3 used a “virtual library” system where you could choose how files would be named and stored and eventually viewed through its interface (e.g., Author-Year-Journal), but they were actually stored under a machine-readable format (long string of numbers; DDC3-VD2383248.pdf); I was never a fan of this opaque system.

Bookends

Bookend-Test2.png

Enter Bookends from Sonny Software , a rather unassuming entrant compared to the more well-known platforms (Mendeley, Zotero, etc.) I had first heard of it through the MacPowerUsers forum and then saw some positive things about it on Twitter. Earlier this week, contemplating the long-term home of my PDFs, I decided to take the plunge and see what the fuss was about.

First off, Bookends on Mac costs $60 - it is a one-time buy with updates lasting for two years (at least). The iOS app costs $9.99 as a one-time buy, and then it is another $9.99/year for enabling cloud-sync. To me, this is a very reasonable pricing structure. Bookends does offer a free trial that limits you to 50 references so you can try it out. But first, is it worth it?

Screen Shot 2019-03-14 at 11.18.19 PM.png

Let me start by saying that Bookends ticks off every bullet point that I mentioned above, and does a LOT more. Starting off, the first thing I did was to investigate how well it can capture a reference from a PDF — this seemed to go very smoothly — Bookends had no problem automatically retrieving information (via JSTOR/Scholar/Web of Life, etc.) for a recently published 2019 article or even one published in 1923.

Ok - so it can perform the basic functionality of a reference manager - what else? Well, the field entries to a reference were quickly editable (refreshingly no lag!), and there were many powerful options for global batch edits. More importantly, the citations’ and reference formats were completely customizable and so was the ability to rename PDF files after importing them. Furthermore, Bookends could sync using iCloud!

Oh my, this seemed rather promising at this point. But - what about iOS? This was where Papers3 excelled. Bookends on iOS did not disappoint. It seemed to be fast, light, and also could fully edit and export citations/references. There was functionality to use customizable search engines (Scholar/Web of Science etc.) for finding articles. Also, you could make notes, highlight, or annotate your PDFs, all of which would sync with the desktop version via iCloud. Furthermore, the app supported split screen view for drag and drop!

With this much potential, I decided to take the plunge. The real test was whether it would be able to handle my 5000+ PDFs and perhaps, even more, pressing: could it port all my existing citations from Papers?