Data Import

Whiteboard-friendly approach

Importing data into Neo4j is very whiteboard friendly -- generally what you draw is what you store. Luckily the Github data is already very graph-like, so modeling is very straightforward.

1. Understand the available data
   • Github as Social
   • Github expanded Social
2. Whiteboard what you'd like to import
3. Generally, import "nodes" then "relationships"

Modify a reference project

1. Pick a reference project
   • github social in scala by akollegger
2. Read through the code
   • notice the strategy for scanning Github
   • identify the conversion from Github data to Nodes + Relationships
3. Fork and modify
   • fork the project
   • add more data from Github
   • add extra in-graph structures
   • add a webapp to proxy Graph Queries
   • add a webapp with Data Visualization

DIY From Scratch

1. pick an Github API Library in your language of choice
2. create a project repository
3. scope your ambition
   • import data, query with neo4j-shell
   • import data, provide web app to proxy Graph Queries
   • import data, provide web app with Data Visualization
4. import strategies
   1. two-phase import - import all nodes, then scan again and create relationships
   2. accounting - import nodes, keeping track of them and creating relationships when possible

References

• Github API Libraries - all the usual languages