Keyring v1.5 & Social Importers v1.4

Yesterday, I released version 1.5 of Keyring, and version 1.4 of the Keyring Social Importers bundle for WordPress. This update moves the Social Importers away from using a postmeta value (keyring_service) and introduces a new taxonomy that keeps track of where posts were imported from. It’s optimized towards management within wp-admin, but you can also use it for front-end queries of your posts. The update for Keyring introduces a new service file for Moves, and fixes a bug in the OAuth2 base service.

The new taxonomy for the Importers is called keyring_services on the backend, and is labeled “Imported From” in the admin UI. It will auto-create itself based on all of the importers installed. You’ll see it within wp-admin under the Posts menu, and will be listed on the “All Posts” listing as well:

Screen Shot 2013-09-15 at 9.10.59 PM

Clicking the name of a service under the “Imported From” heading will filter the posts list by that service (e.g. Twitter). The main reason that the taxonomy is exposed through the admin UI is so that you can tweak the slugs if you’d like to. I noticed that on my install, I’d already used things like ‘twitter’ and ‘foursquare’ as tags, and so they had claimed the namespace for that slug. WordPress’ shared terms are annoying like that :). So, if you’d like to use the slugs of source services in URLs, you might want to rename them:

  1. Go to Posts → Tags
  2. Search for and rename the slug for each of the services (e.g ‘twitter’, ‘foursquare’, ‘flickr’). Name the slugs something like ‘twitter-3’
  3. Go to Posts → Imported From and rename the slugs for each service to the “clean” version (without a ‘-2’).
  4. Optionally go back to Posts → Tags and rename those tags again back to the -2 versions.

As part of this change, you’ll want to update any previous posts that you imported to using the new taxonomy. I’ve included a quick and dirty script to do this. It’s called migrate-keyring-postmeta-to-taxonomy.php and can be found in the root of the plugin. To use it, you need to move it to the root of your WordPress install, and then you can just access it through your browser. It’s likely that it’ll run out of memory or time out, but it’s written in a way that you can just run it over and over again until it finishes cleanly. On my server, once it was finished and produced no output, Chrome decided to display a “friendly” error message instead of anything useful. Once that’s done, your existing posts should all be converted over to using the new taxonomy, and there should be no more postmeta entries for keyring_service.

If you’re doing a clean import, I recommend doing it without auto-import enabled, and then once you’ve fully imported everything, enable auto-import and let it run from there.

Import DokuWiki pages into WordPress

I used to maintain a wiki full of personal ideas using DokuWiki, but at some point just gave up with wiki syntax entirely. A few months ago when I was moving all of my sites and content over to a new server and trying to consolidate things as much as possible, I decided to import all of that old content into a WordPress install (which was actually a single site within the same Multi-site install that runs Dented Reality). I ended up writing the following super-rough script to just scrape the contents of the pages and throw them into WordPress. Scraping the pages meant that I could get the actual output of all plugins etc, and also get full links between pages.

(more…)

The All-New LiveJournal Importer for WordPress

Over the past few weeks, I have been working on a new importer for people who use LiveJournal, but would like to switch over to WordPress. With LiveJournal laying off a bunch of employees, it seemed like some people might prefer to move to a platform where they had a bit more control over their own content, rather than relying on another company to handle it for them. I decided that my measure of success would be that it needed to be capable of importing Guav’s entire journal — comments and all, without error.

As it turns out, it’s been quite a project. LiveJournal’s API is, shall we say, “challenging” to work with, and the sheer size of Guav’s journal (over 3,700 posts and nearly 200,000 comments) meant that I kept running into time, memory and database limits that would crash the importer. After a lot of back and forth with Guav though, I’m happy to present the new importer (find it under Tools > Import > LiveJournal), sporting the following features:

  • Just enter your LiveJournal username and password and you’re ready to go
  • Via the API, it connects directly to LiveJournal and imports all of your posts,
    • Posts marked as “Friends Only” are assigned a password within WordPress,
    • Posts marked as “Private (you only)” are marked as Private within WordPress, which means that only authors on your new blog can read them,
    • lj-cut tags are converted to the WordPress equivalent, the <! — more — > tag,
    • lj-user tags are converted to normal links, and have class=”lj-user” attached to them to make it easier to style them if you like,
    • Tags are imported properly,
    • If you closed the comments on a post in LiveJournal, then they’ll be closed in WordPress as well,
    • Lots of the “meta” information related to posts is also imported using WordPress’ Custom Fields feature. You could then use these values to reproduce some of LiveJournal’s functionality within your new theme if you like. The fields imported are:
      • If your post contains adult content (lj_adult_content),
      • Your current co-ordinates and location (lj_current_coords and lj_current_location),
      • Your current mood (lj_current_mood),
      • Current music (lj_current_music),
      • Your userpic keyword (lj_picture_keyword)
  • Next up, all of your comments are also imported,
    • Threading is preserved, so replies to other comments show up successfully (provided you enable that feature in WordPress),
    • The “subject” of each comment is included as the first line of the comment itself, because WordPress doesn’t have a comment title/subject value,
    • LiveJournal users get links back to their LiveJournals,
    • Your own comments are linked to your WordPress account, and are linked back to your new WordPress blog,
    • Anonymous users are labeled as “Anonymous”,
    • “Screened” comments on LiveJournal are imported as “Unapproved” within WordPress, so you can decide what to do with them

So there you have it, a brand new, shiny LiveJournal importer. This should be bundled with the 2.8 release of WordPress (it’s available right now on WordPress.com), and will be available for everyone. As I mentioned, it’s been tested with one single, very large blog (and a few smaller test ones), but if you find anything wrong with it, please file a bug on the WordPress Trac!

Huge thanks again to Guav for helping with the testing of this thing (maybe now he can migrate over to WordPress as well)!