BeauLebens.com

An aggregation of Beau on the internet

Menu

Skip to content
  • Blog
  • Archive
    • Posts
      • Tweets
    • Images
      • Flickr
      • Instagram
    • Links
      • Delicious
      • Instapaper
    • Places
      • Check-ins
      • Trips
  • Explore
  • Projects

#data

Data Viz Project

http://datavizproject.com/
  • #charts
  • #data
  • #information visualization
  • #infoviz
  • #reporting

http://datavizproject.com/

Really cool library of visualizations with standardized colors to remove unnecessary visual variations. Love it.

9:14 am, October 1, 2017

Scrapinghub | Web Crawling Platform & Data as a Service

https://scrapinghub.com/
  • #crawling
  • #data
  • #scraping

Scrapy Cloud, our cloud-based web crawling platform, allows you to easily deploy crawlers and scale them on demand – without needing to worry about servers, monitoring, backups, or cron jobs. It helps developers like you turn over two billion web pages per month into valuable data.

Source: Web Crawling Platform & Data as a Service | Scrapinghub

12:23 pm, May 20, 2017

People & Places

https://beau.blog/2017/01/people-places/
  • #data
  • #geo
  • #keyring
  • #keyring social importers
  • #people
  • #places
  • #plugin
  • #taxonomies
  • #wordpress

Over the years, I’ve been working on a system to aggregate data that I publish to other social networks/sites back into my control, on my own WordPress install. Thus far, that has resulted in the creation of Keyring (plugin) to provide an abstracted interface to all of the web services I’m interested in, Keyring Social Importers (plugin) to do the basics of importing the data from different places, and Homeroom (theme) to display it all. Today, I’ve been working on a system that will detect people who are mentioned in an interaction, and link them across posts using a custom taxonomy. It does the same for physical locations, so I’ve called it People & Places.

Essentially, this plugin is just a pair of custom taxonomies, with some specific ways of referring to things. Pretty basic. It gets more interesting though when you update Keyring Social Importers to the trunk version, which will now work in tandem with People & Places to link everything up. I wouldn’t recommend it on a production site just yet — there’s a lot of rough edges still.

When KSI is pulling in content from each service (currently looking at Twitter, Instagram and Foursquare), there’s a new block of code that makes sure People & Places is available, and then looks for certain pieces of data. If it finds them, it bundles up the details, and passes that along in the import process. When posts are actually inserted, it will attempt to link up that post to the People/Places it found. If the People already exist, then they’ll just be linked, in the same way tags work. If they don’t exist yet, then a new Person entry will be created, and that will be used.

I plan to add in a basic term-merging function, so that you can manually (maybe automatically?) identify “duplicate people” across different networks, and intelligently merge their entires (re-linking any posts involved), so that you build up a single, combined view of your interactions with a particular person. I envisage some interesting possibilities with the archive pages for these taxonomies, and that over time it will build a really interesting dataset of your interactions, the places you physically go, etc.

I’ll probably still move the code around a bit, and there are definitely some bugs around duplicates and handling things across different networks, but it seems to be working so far. This is also probably the time to figure out a decent way to allow re-processing of imported data from the raw copy that the importers save in postmeta. Installing this new code will start gathering data on new imported entries, but won’t go back and do the same on all the posts you’ve already got. Rather than deleting all that data and re-importing/processing everything, I’d like to have a simple way to re-process the raw data that’s already stored locally.

8:28 pm, December 31, 2016

Swarm.js

http://swarmjs.github.io/about/
  • #collaboration
  • #data
  • #javascript
  • #mobile
  • #offline
  • #sync
  • #syncronization
  • #webapps

Swarm.js

Swarm is a reactive data sync library and middleware. Swarm synchronizes your app’s model automatically, in real time.

Saved on Delicious 3:35 pm, October 19, 2014

artoo.js

http://medialab.github.io/artoo/
  • #data
  • #javascript
  • #scrape
  • #scraping
  • #spider

artoo.js

The client-side scraping companion

Saved on Delicious 4:14 pm, June 21, 2014

WP Test

http://wptest.io/
  • #data
  • #import
  • #plugin
  • #testing
  • #theme
  • #wordpress

WP Test

A fantastically exhaustive set of test data to measure the integrity of your plugins and themes.

Saved on Delicious 12:57 pm, March 19, 2013

D3 – Data Visualization

https://github.com/mbostock/d3/wiki
  • #charts
  • #data
  • #javascript
  • #visualization

D3 – Data Visualization

Beautiful JS-powered data visualization/manipulation library.

Saved on Delicious 11:33 am, June 14, 2012

41Latitude – Google Maps & Label Readability

http://www.41latitude.com/post/2072504768/google-maps-label-readability
  • #analysis
  • #data
  • #google
  • #maps

41Latitude – Google Maps & Label Readability

Really awesome analysis of why Google Maps are easier to read than either Bing or Yahoo. Looks at labels, clustering etc.

Saved on Delicious 9:41 pm, December 3, 2010

A Revised Taxonomy of Social Networking Data

http://www.schneier.com/blog/archives/2010/08/a_taxonomy_of_s_1.html
  • #data
  • #identity
  • #privacy
  • #security
  • #socialmedia
  • #socialnetworking

A Revised Taxonomy of Social Networking Data

Looking at different types of data involved in social networking and how to secure/control it all.

Saved on Delicious 4:24 pm, August 10, 2010

Infochimps

http://infochimps.org/
  • #data
  • #database
  • #datamining
  • #datasets
  • #repository
  • #research
  • #statistics
  • #visualization

Infochimps

“Infochimps is an open catalog and marketplace for the world’s data. You can share, sell, curate, and download data about anything and everything.”

Saved on Delicious 5:38 pm, March 18, 2010

Post navigation

← Older posts

People

  • Erika Schenck (1,816)
  • Helen Hou-Sandi (194)
  • Automattic (177)
  • Scott Taylor (132)
  • Kelly Hoffman (131)

Categories

  • Uncategorized (28,843)
  • Personal (9,315)
  • Posts (304)
  • Techn(ical|ology) (192)
  • Projects (77)

Tags

  • read (3,919)
  • wordpress (624)
  • sanfrancisco (421)
  • automattic (394)
  • photo (392)

Year

  • 2025 (225)
  • 2024 (1,014)
  • 2023 (953)
  • 2022 (819)
  • 2021 (906)
Powered by Homeroom for WordPress.
 

Loading Comments...