Drumbeat/MoJo/hackfest/berlin/projects/nwsmkr

Project Name:

nwsmkr

Project Lead(s):

Any and all of us who are interested in building a solid storytelling and visualization tool for newsrooms and journalists

Project Goal:

Aside from having fun, getting to know each of you in person, helping each of you build and create and touring the city...

To come away with a working web form that searches against the LittleSis.org API, and marries it with user input to generate a JIT javascript visualization and an embed code to display on a website.

OR

Pick the brains of everyone available to me, spend the week learning what skills I need to make this happen in the future and come away with a technical roadmap that I can follow so I can create this.

Milestones to reach project goal:

Brainstorm with group to see what aspects have been overlooked, are missing or could become an obstacle

Create web form that allows for dynamic options and submission of variables

• How many primary connections
• Size of visual
• Title of visual
• Explainer text of visual

Search LittleSis.org API for potential variables

• Structure query request of LittleSis.org database
• Return results to user with options to select profiles for inclusion into visualization
• Leave place for hooks that will allows writing of data to LittleSis.org database

Write a JSON data data structure

• Data structure will contain variables for the user and LittleSis.org data.
• Data structure will be added to the JIT javascript library

Design and code rapid deploy template

Needs:

I'd love to have someone with Python/django/sql skills and familiar with pulling APIs into a JSON structure.

At this point I'm not focusing on how the user interface would look, so there is an opportunity there for a designer or HTML/CSS lead who wanted to jazz up what likely would be a pretty plain looking form to start.

I'd love to recruit a journalist or two to lead the storytelling effort, and -- to paraphrase Jordan -- make sure the content sings whether it's a series of stories or simple labels and annotations that lead the user to the most important aspects.

Additional information:

Data Source

Thanks to members of #hacktoberfest I have been able to chat with Kevin Connor and Matthew Skomarovsky who co-created LittleSis.org, which bills itself as "An involuntary facebook of powerful Americans, collaboratively edited and maintained by people like you…"

Techically speaking, LittleSis.org is a relational wiki database designed to track information on powerful people, organizations, and networks -- more than 80,000 of them -- that has strict sourcing requirements, a strong permissions system, and full modification tracking that guard against editorial abuse, help users find original sources, and make the editing process transparent.

The LittleSis API makes it easy to retrieve data entered into the system for display on other sites, and a bookmarklet allows selected user to add information to the database directly from a news story.

Best of all, LittleSis is free and open-source.

Visualization Library

The JavaScript InfoVis Toolkit provides tools for creating Interactive Data Visualizations for the Web.

Link for more info:

1. moznewslab final presentation is here.

Link for demo:

Coming soon...

Source code:

GitHub Repo -- at least the link to it -- is here. Username is Chris L Keller.

Extending and applying nwsmkr: A case study

Project description and rationale is here [Google doc] -- Jordan will put the full text directly into the Wiki soon!

Presentation at #hacktoberfest

Final presentation/demo at the Berlin hackfest

Project Status

• Currently working features include:
  

1. Demo mind map of the Wal-Mart article example using d3.js
   
2. Querying a name in Little Sis and returning a network diagram
   

• The project is currently capable of demoing some features. It's not fully functional yet, but has been really useful for understanding how it might be applied to newsgathering.
  
• The project currently functions in the context of inputing search terms (people, organizations), outputing a basic network digraph
  

Collaborators

In addition to Saleem, Chris and Neil who were also working on extending nwsmkr, these folks helped with this sub-project:

• Tathagatha - Demonstrated how to scrape names and organizations from an article
• Matt Haynes - Wrote javascript for adding names to network diagram, creating diagram from a person's Little Sis info
  

Next steps

- From here we would like to:

• Migrate personal repo to organization-based repo.
• Demo how you might plug user-added data back into the database using the visual features of d3.js. Is this a useful way of interacting with the data? Is drawing a link or clicking to add a note easier for the user than filling out a form on Little Sis?
• Find out if anyone has worked on turning network map into an API (so you can build off of it)
• Start thinking about how to tackle the filtering problem -- this will only be useful if you can sift through the noise and pull out the pertinent information
• Reach out to Jonathan Yankovich (@tronathan), a javascript developer whose goal is to write a "wrapper library" for D3 that would let people run all kinds of common graphs and charts, and then allow them to extend in the future if needed.

Lessons learned

• Putting data back in to the Little Sis database needs to be just as easy as pulling data out. I don’t know enough about the API to know if this would be possible. If it isn’t yet possible, I’d love to see it.
• Networks get very complex very quickly. It’s the “seven degrees of Kevin Bacon effect,” only on Little Sis it happens even faster. In two or three steps you get to President Obama, and soon everyone is connected to everyone (and everything is connected to everything).
• The usefulness of using databases of networks as a newsgathering or information seeing tool is a real problem that needs to be explored. Because the rabbit hole is so deep and dark, to make this kind of tool useful to a journalist or a news reader, filtering the information for the most relevant parts is going to be essential. How can you pull out the links and relationships that are meaningful?
  

Places where this project might be tested include

• An investigative journalist telling the story of campaign contributions with data from OpenSecrets.org
  
• A reader trying to understand the broader context of an article -- for example, jumping in to coverage of an ongoing topic and filling in the background; discovering the political and personal connections of people quoted in a story (are they reputable?); alternative to just Googling the names of people or organizations you don't know.
• A journalist writing a follow-up story on an issue could use this to surface ideas about who to call next, what organizations to research, how to expand the story, etc.