soam's home

home mail us syndication

Archive for video

Escape From Amazon: Tips/Techniques for Reducing AWS Dependencies

Slides from my talk at CloudTech III in early October 2012:

Escape From Amazon: Tips/Techniques for Reducing AWS Dependencies from Soam Acharya


Talk abstract:

As more startups use Amazon Web Services, the following scenario becomes increasingly frequent – the startup is acquired but required by the parent company to move away from AWS and into their own data centers. Given the all encompassing nature of AWS, this is not a trivial task and requires careful planning at both the application and systems level. In this presentation, I recount my experiences at Delve, a video publishing SaaS platform, with our post acquisition migration to Limelight Networks, a global CDN, during a period of tremendous growth in traffic. In particular, I share some of the tips/techniques we employed during this process to reduce AWS dependence and evolve to a hybrid private/AWS global architecture that allowed us to compete effectively with other digital video leaders.

Flattering Streaming Media Coverage for Our Video Platform

Streaming Media writes that OVP (Online Video Platform) is a pretty crowded space with more than a hundred vendors. They then list the top ten platforms and yes, we’re in there. Here’s what they say:

The OVP formerly known as Delve (until Limelight bought it) stands out for its user experience and workflow. The user interface was designed to be friendly and to make it easy to accomplish tasks. When a user uploads a video, he or she can update its metadata and set the channel even before the video is fully uploaded. That’s a timesaver few can match. It also offers strong analytics and APIs.

The platform’s Video Clipper tool makes it easy to shorten videos and works well in combination with Limelight’s real-time analytics. If users see that viewers are routinely quitting a video at a certain point, they have the option of ending the video there.

The OVP was acquired by Limelight in August 2010. Rather than focusing on just the mobile experience or just analytics, the Limelight Video Platform focuses on offering the best end-to-end user experience. Since it’s now under the same roof as a top CDN, it’s able to offer tie-ins that others can’t. Users gain from functionality such as player edge scaling, which the video platform is able to offer through low-level access to the CDN. Users also get to use developer APIs that aren’t open to the public.

Given that we started relatively late in this space (the company had started to pivot away from semantic video search to the platform SaaS approach just prior to my coming on board), this recognition is particularly heartening. It has been a monumental amount of work and accumulation of many battle scars for all of us. Full credit to Alex, Edgardo and the rest of the team!

Being committed to a startup means you have to be prepared to do anything and everything. That and my own role(s) at Delve/Limelight mean involvement with pretty much all system aspects, especially the backend. I am particularly chuffed at the repeated mentions of analytics since a lot of my own blood, sweat and tears go into it.

Okay, better stop now before this starts reading like an acceptance speech 🙂

Video Clip Lengths

In a NYT article, Rise of Web Video, Beyond 2-Minute Clips, Brian Stelter writes:

Video creators, by and large, thought their audiences were impatient. A three-minute-long comedy skit? Shrink it to 90 seconds. Slow Internet connections made for tedious viewing, and there were few ads to cover high delivery costs. And so it became the first commandment of online video: Keep it short.

I recall coming across this phenomenon in 1997 and 1998 while doing research work into characteristics of web video stored on the web at that point in time. Here’s an interesting graph from the paper I wrote on the subject:

Web Video Lengths (1997)

Web Video Lengths (1997)

The number of videos on the web were relatively small and their sizes could be measured in seconds. 90% were 45 seconds or less. The graph is capped at around 2 minutes for maximum length although I did find outliers that were longer.

What I found interesting, however, in a followup study was that if you took away the bandwidth chokepoints, video lengths ballooned. I was studying the video access patterns of a Video On Demand experiment at the Lulea University in Sweden – the setup here was over a dedicated high speed network, effectively removing slow access as a determinant of behavior. Specifically:

Since 1995, the Centre for Distance-spanning Technology at Luleå University (CDT) has been researching distance education and collaboration on the Internet [17]. Specifically, it has developed a hardware/software infrastructure for giving WWW-based courses and creating a virtual student community. The hardware aspects include the deployment of a high speed network (2-34 Mbps backbone links) to attach the local communities to the actual University campus. The campus is also connected to the national academic backbone by a high speed 34 Mbps link [13] with student apartments being wired together with the rest of campus via 10 or 100 Mbps ethernet.

The following graph shows the distribution of video lengths for the files used in the system:

VoD Video Durations

The mean duration of these files were around 75 minutes or so. This finding hinted that as videos grew in popularity and infrastucture hurdles fell away, video durations would increase. From the original NYT article:

New Web habits, aided by the screen-filling video that faster Internet access allows, are now debunking the rule. As the Internet becomes a jukebox for every imaginable type of video — from baby videos to “Masterpiece Theater” — producers and advertisers are discovering that users will watch for more than two minutes at a time.
“People are getting more comfortable, for better or for worse, bringing a computer to bed with them,” said Dina Kaplan, the co-founder of

Ms. Kaplan’s firm distributes dozens of Web series. A year ago all but one of the top 25 shows on her Web servers clocked in at under five minutes. Now, the average video hosted by Blip is 14 minutes long — “surprising even to us,” she said. The longest video uploaded in May was 133 minutes long, equivalent to a feature-length film.

Interested by this, I took a look at the duration of the videos hosted by Delve. This is based on data a couple of weeks old, so this is not representative of the latest trends. However, I found the average video duration to be a little under 6 minutes. However, within this I found definite disparities between publishers. Our top 25 publishers (by video duration) had videos that were a little under 25 minutes on average. This indicates mixed video use by our publishers. While some are still sticking to shorter videos, a significant number are definitely taking full advantage of long form clips – one of the largest videos is around 12 hours in length!

It’ll be interesting to see how these trends hold over the next year or so.

Delve Reviewed On InformationWeek

Fritz Nelson reviews our publishing system on InformationWeek and posts a video walkthrough with Edgardo, our VP of Products:

Having spent much time of late working on our analytics, I was particularly gratified by Fritz’s comments:

I also liked Delve’s reporting. Most of the video hosting solutions I’ve worked with tend to be a bit sparse on detail — in fact, I’ve found this to be a glaring weakness in most every platform. It’s fine to know how many views a video got, or the length of viewing time, but being able to hone in on viewership numbers based on syndicated player, for example, or time of day is becoming increasingly important. While I only got a cursory look at the reporting, it seems quite robust, and as with many systems, you can pull its data into something like Omniture.

More to come here, stay tuned!

Online Video And The Future Of Journalism

Interesting observation by Arianna Huffington in her statement to the US Senate Hearings on “The Future Of Journalism” held last week (May 6, 2009):

No, the future is to be found elsewhere. It is a linked economy. It is search engines. It is online advertising. It is citizen journalism and foundation-supported investigative funds. That’s where the future is. And if you can’t find your way to that, then you can’t find your way.

Online video offers a useful example of the importance of being able to adapt. Not that long ago, content providers were committed to the idea of requiring viewers to come to their site to view their content — and railed against anyone who dared show even a short clip.

But content hoarding — the walled garden — didn’t work. And instead of sticking their finger in the dike, trying to hold back the flow of innovation, smart companies began providing embeddable players that allowed their best stuff to be posted all over the web, accompanied by links and ads that helped generate additional traffic and revenue.

This dovetails into the type of work we’re trying to accomplish at Delve. It’s nice to think that in some small way, we might be a part of the solution and not the problem.


I’ve been quite a fan of Roku’s set top box since buying it for Netflix’s Watch Now service. The ease of installation and use, a small form factor and bargain basement pricing clearly marks it out as a winner. The video quality is very good as well although my experimentation has been limited to 480p, not HD. Though limited to two channels at the moment (Netflix and Amazon), Roku has plans to add more to its lineup later this year. Thus far, Roku has been mum on exactly what those channels might be. However, the interviews by the top brass over there, particularly Tim Twerdahl, VP of Consumer Products, have revealed some tantalizing details about their plans, philosophy and how well they are doing.

From trenderresearch:

“We are running at full tilt, selling them almost as fast as we can make them,” he said. Tim did not disclose numbers, but says “we are absolutely making money on the hardware” and tells me unit sales are well into the “hundreds of thousands.” Surprisingly, Tim also downplayed how much money Roku makes on content sales either through revenue-sharing or referral fees, saying “We really make money on the box… occasionally there are bounties for bringing new (eyeballs) to the content owners.”


Tim tells me that the Roku player takes advantage of the new cloud computing model that allows the box to be a “thin client.” The Roku player only needs to stream and buffer about 4 minutes of video at a time, eliminating the need for a large hard drive. It does not need encryption inside the box, instead leveraging the DRM protections inherent in the connectors— similar to what a DVD player does.

From Contentinopole:

Contentinople: Can you tell me more about the content providers you’re talking to — like MTV Networks or Comedy Central or Hulu?

Tim Twerdahl: I can’t obviously talk about any of those specific deals. But I’ll tell you philosophically where we’re going.

We look at content for our boxes as this matrix: You have various kinds of content — TV shows, movies, YouTube. One of the key cells that we don’t have today in that matrix is ad-supported TV. We will have ad-supported TV this year.

User-generated clips that are free or ad supported is something we’re focused on. Subscription-based music and free music are things we’re looking at.

Rounding out that content matrix is what we’re focused on now. This summer, we’ll be releasing an SDK [software development kit] publicly where anybody can create a channel, and we will have a marketplace for those channels so that our users can choose to add the channels they’d like to to the box.

We strongly believe that we don’t have a monopoly on every great idea for what would make sense to put on the TV, and so this SDK will really let the community help define that for us.

Bottom line: though intially, this box was marketed as a hardware based Netflix Player, there is much more potential here for programming including music only services. Worrying news for cable companies and other set top box manufacturers.