Barrel - Simple Key Value Store

I made a really simple gem called Barrel. You can use Barrel with Rails to store things. It will keep them and give them back later. Especially good for long running calculations.

Installation

Add this line to your application's Gemfile:

gem 'barrel'

And then execute:

$ bundle
$ rails generate barrel:install

Usage

Barrel is so simple:

Barrel.store 'Total Monkeys', '42'
Barrel.find 'Total Monkeys'
# => '42'

AWS S3 Copy File Size Limit

Last year, I rolled together a custom video platform for WorshipU using AWS. It includes an admin for uploading videos to S3, transcoding to multiple formats using Elastic Transcoder, delivery using Cloudfront, and video.js to play the videos in the browser.

The admin allows content managers to upload a bunch of videos at the same time to a designated upload folder in the S3 bucket, and then associate the videos with lessons at a later time. Upon association, the video is moved to a folder specific to the lesson, based on the lesson's id, quality, and encoding:

310/720.webm

WorshipU is a Rails application, so naturally I used the official AWS ruby gem for accessing their API. To move the video from the uploads folder to its respective lesson folder, I used the move_to method for S3Objects. Behind the scenes, move_to performs a copy and then deletes the original.

# get video from S3 uploads directory
video = bucket.objects[lesson.tmp_url]

# move the file to its new location
video.move_to("#{lesson.id}/original.mp4")

I've been really happy with this stack, and it worked great for about a year. Then recently, the content manager started uploading videos with much larger file sizes, and the background task that processes videos started failing. That's how I discovered that S3 has a 5gb file size limit on copying.

It looks like newer versions of the gem may address the file size limitations automatically. I couldn't find any information on it, but if you are using aws-sdk-v1, like me, then there is a sparsely documented way around this using the copy_from method. I could only find this pull request discussing it, but it allows you to do a multipart copy by passing respective parameter. It also request you to pass the length of the item as well.

# get video from S3 uploads directory
video = bucket.objects[lesson.tmp_url]

# create temporary object in new location
copy = bucket.objects.create("#{lesson.id}/original.mp4", "tmp")

# multipart copy from the uploads directory
# to the new object
copy.copy_from(
  video, 
  content_length: video.content_length, 
  use_multipart_copy: true
)

# delete the file in the uploads directory
video.delete

This fixed the problem perfectly for me, although with a few more steps. It would have been nice if I could have just sent the multipart copy option to the move_to function.

Blossom API Gem

tl;dr

I created the blossom_api gem for Blossom's API.

gem 'blossom_api'

...then:

$ bundle

...and finally:

my_project = BlossomApi::Project.new(
  organization_id: 'xxx',
  project_id: 'xxx',
  access_token: 'xxx'
)

my_project.create_card 'New feature', 'Description of the feature'

...more features to come.

The rest

A team I work on has been testing out using Blossom for our project management over the past couple weeks. So far, I have really enjoyed how it both tailors to developers and looks good while doing it. I'd definitely recommend it.

Leading up to making this switch, our development team's structure had really be compromised. There was no central point of entry for the other departments to make requests, or ask for solutions. In order to try to clear up communication, keep everyone informed, and move effectively, it made sense to me that only developers should have Blossom accounts. This degree of separation is not meant to be mean, it's just not ideal for other departments to be muddling about in our project management software. Who better to make decisions about priority, assignment, and implementation than the people actually developing?

To help bridge the gap caused by this, I created a system for other staff to be able to submit requests to an "Inbox" project through a form on the staff portal. Then, I can go through and assign requests in Blossom appropriately. The ability to create cards is available in the gem.

I also created a calendar interface to the Blossom API so that other staff can see what we are working on, our project completion of tasks, etc. I haven't extracted that in to the gem yet, but I plan on doing a lot more with it.

First Friday Jams - Caribou Playlist

Last month, Caribou released a playlist on YouTube containing over 1000 songs that have influenced him over the past couple years. It's really incredible, and diverse. He knows music. Hope you enjoy it as much as I have.

I should mention there is also a Spotify version of this playlist, but it's not near as comprehensive.

Caribou also released a great album last year called Our Love.

Open Source Christmas

This month, I've been participating in the 24pullrequests challenge. The idea is that you use open source projects all year for free. So, why not spend December giving back?

I've been able to submit a pull request to an open source project for each day this month up until Christmas Eve. That's 24 pull requests to 8 different open source projects, including:

Check it out!