Drumbeat/MoJo/hackfest/berlin/projects/LikeLines
Contents
About the Project
LikeLines unlocks user-sourced video and serves as a building block for rich news story telling. Interesting bits of a video emerge naturally through community interaction with the video, allowing for video navigation and retrieval at the fragment level.
Team
Project Lead: Raynor Vliegendhart
GitHub: ShinNoNoir
Twitter: ShinNoNoir
Project Status
- Currently working features currently only includes the GUI.
- The project is currently capable of showcasing how the user can interact with the video.
- The project currently should function in any context (it's a JavaScript library), but only shows predefined heatmaps for a small set of videos.
Collaborators
The following folks helped with this project:
- Dan Shultz: Discussion about timecode comments on video pages as an additional source.
- Tathagata Dasgupta: Provided a link to heatmap.js.
- Cole Gillepsi: Provided a link to Mozilla Dev Derby 2010's HTML 5 Video Voting project.
- Jordan Wirfs-Brock: Discussion about using LikeLines for editing and mixing of raw videos.
- Cody Shotwell: Discussion about the UI, about alternative ways of visualizing the time, number of likes and number of clicks, about delay of the intended liked points and the actual clicking of the button.
Next steps
From here I like to:
- Work on points 2 and 3. That is, support more video sources than just YouTube and that it generates user-feedback based heatmaps.
- Deploy a small example video front-end site and get people to use the LikeLines player. Gather usage statistics and refine algorithms in use.
- Hook it up to the Meta Meta Project.
Places where this project might be tested include:
- Any particular video website to generate usage metadata.
- Any website that summarizes a collection of videos (e.g., Juan Gonzalez's Tribal Mix dashboard idea). LikeLines metadata can help to choose which parts of the video should be included in the visual summary.
- Video fragment recommender systems. (This requires that LikeLines metadata can be tied to a (virtual) user and that the metadata is stored securely to ensure privacy.)
Original Big Goal for MoJo Hackfest
Create a fully working prototype of LikeLines that generates navigation heatmaps based on user interaction (instead of displaying static heatmaps) and expand support to non-YouTube video players.
Additionally, I'd like to help others where I can.
Key steps toward goal
- Finalize first version of the (server-side) API and decide on some storage format
- Add support for e.g. Popcorn.js
- Rework the GUI code
Pending needs
Some guidance with Popcorn.js and CSS might speed up the development process.
Offered skills
- Python
- peer-to-peer (P2P)
- little bit of information retrieval (IR)
Project updates
Day 1:
- Created a GitHub repository and this wiki page
- Decided on the LikeLines API
Day 2:
- Created a dotCloud account for the server-side component
- Created another GitHub repository for the server-side component
- Wondering about how to compute the heatmap (interpolation and smoothing of liked points)
- Discussion with Dan: Use of comments with timecodes on a video page as an additional source for heatmap generation
Day 3:
- Finished the boilerplate for the server side component
- Interesting discussion with the Guardian: Can LikeLines be used with live video? Should the LikeLines heatmap be live?
- Feedback from Cole: HTML 5 Video Voting from last year's Mozilla Dev Derby (2010) uses a similar idea to LikeLines
- Feedback from T: Look into using the heatmap.js library for generating the heatmap
- Feedback from Jordan: LikeLines can be used to take a bunch of raw videos and ease the process of editing and mixing them (similar idea to originally proposed as a use case in the MoJo Drumbeat submission)
Day 4:
- Final wikification of the #hacktoberfest event
--post #hacktoberfest--
Day 4+1
- More cleaning up of the wiki
Day 4+3
- Found out that Hulu also introduced heatmap navigation to their player. However, Hulu chooses to stick with Flash and not HTML5. LikeLines differentiates itself by being open source and positioning itself as a building block that can be used by anyone.
LikeLines API
The LikeLines API uses JSONP to allow cross-origin access.
Protocol Draft
- /heatmap?video=<url>[getkey=<1|0>]
Returns heatmap metadata for the specified video and optionally an interaction key for sending user interaction data to the LikeLines metadata server. - /events?video=<url>&key=<interaction_key>&events=<events>
Sends a list of high level events to the LikeLines metadata server. - /like?video=<url>&key=<interaction_key>&t=<timestamp>[&old_t=<timestamp>]
Store a timecoded "like" to the LikeLines metadata server. When old_t is specified, the old "like" at that timepoint is removed.
Events
User interaction with the video player are represented by fragments played. A played fragment is denoted by a timestamp of when the fragment was played and an interval of timecodes.
An example is shown below:
t=0 [0..20],
t=40 [20..40],
t=60 [30..50],
t=80 [120..140]
In this example, the user watched the video for 20 seconds, after which he paused the video for 20 seconds and continued to watch the video at t=40 for another 20 seconds. The user then rewound to the 30th second of the video and watched another 20 seconds, after which he skipped to the 2nd minute of the video and watched another 20 seconds for the last time.
