Sign inSign up

eagleoutice/cran-pkg-db-neo4j

By eagleoutice

Updated about 2 months ago

Dataset of the History of the CRAN Package Repository

Image
Data science
1

143

eagleoutice/cran-pkg-db-neo4j repository overview

CRAN package graph (crawlr)

A queryable graph of the CRAN R package ecosystem: packages, versions, functions, constants, dependencies, call graphs, CRAN task views, source repositories, and authors/maintainers. Built with Neo4j Community Edition and served through Neo4j Browser -- a web UI for running Cypher queries and exploring the graph interactively. No setup beyond docker run: the database is already imported and ready to query.

This image, this page, and the Neo4j Browser configuration it ships with (read-only landing, auto-rendered example graph, the 23-query reference table) were put together with AI assistance.

Quickstart

docker run -d -p 7474:7474 -p 7687:7687 -e NEO4J_AUTH=none eagleoutice/cran-pkg-db-neo4j:latest

Then open:

http://localhost:7474/browser/?connectURL=bolt://localhost:7687&db=cranpackages&preselectAuthMethod=NO_AUTH

and click Connect (leave username/password blank -- auth is disabled server-side). It lands directly in the cranpackages database, already in read-only mode (write queries are rejected server-side, not just hidden in the UI), and three things render automatically:

  1. Schema overview -- every node label and relationship type at a glance.
  2. ggplot2's own transitive call graph (latest version, bounded to depth 10) -- a real node-link diagram of actual functions calling each other, not just the abstract schema.
  3. All 23 example queries, as a table of description + Cypher, so there is nothing extra to find or open even if you have no access to the source repo -- copy a row's Cypher into the query bar to run it yourself (note: Neo4j Browser's grid wraps copied string values in "...", strip those before running).

Graph schema

Node types: Package, Version, Function, Constant, TaskView, Repository, Person. Relationships: HAS_VERSION, NEXT_VERSION (chronological), DEPENDS_ON, DEFINED_IN (from both Function and Constant), CALLS, IN_TASK_VIEW, HOSTED_AT, AUTHORED_BY, MAINTAINED_BY. Exported functions/constants carry an extra Exported label, so Neo4j Browser's automatic per-label coloring shows them differently from internal ones with no style configuration needed. Function and Constant IDs are readable (pkg::name::firstSeenVersion), not opaque hashes, and are content-deduped -- the same node is reused across every version whose implementation is byte-identical. Person nodes (authors/maintainers) are extracted from free-text DESCRIPTION fields on a best-effort basis -- the same real person under different name/email spellings may appear as distinct nodes.

CALLS resolves a callee name against every Function in the whole dataset sharing that name, not scoped by package/import -- deliberately ambiguous, so masking/collision patterns (e.g. dplyr::filter vs stats::filter) show up as fan-out in the graph rather than being silently resolved away.

Downloading the raw graph data

The same image also bundles the raw CSV export (the exact files that were bulk-imported to build this database: packages/versions/functions/constants/.../calls CSVs, plus import.sh, queries.cypher, constraints.cypher) at /csv-export -- pull it out without ever starting Neo4j:

docker create --name crawlr-tmp eagleoutice/cran-pkg-db-neo4j:latest
docker cp crawlr-tmp:/csv-export .
docker rm crawlr-tmp

(docker rm is just cleanup -- not required for the copy itself, since docker cp reads from any container, running or not.) To load the CSVs into your own Neo4j instance: cd csv-export && ./import.sh (needs neo4j-admin on PATH, run before the target database has ever started), then apply constraints.cypher.

Known issues

  • Connection.MissingDatabaseError "No active database" can appear on one of the three auto-rendered frames -- a timing race in Neo4j Browser's own connection setup, not something this image's configuration can prevent. Harmless and one-time: paste the query back into the query bar and run it again.
  • Copying a cell out of the examples table includes surrounding "..." quotes on string values -- that's Neo4j Browser's own copy-button behavior, strip them before running.
  • URL-valued properties (e.g. a Version's cranUrl) render as clickable links when returned as a query-result column, but not in the "Node details" side panel from clicking a node in the Graph view.

License

Public CRAN package metadata. This image ships without a password (NEO4J_AUTH=none above) -- set -e NEO4J_AUTH=neo4j/<your-password> instead if you're exposing it beyond localhost.

Tag summary

Content type

Image

Digest

sha256:13e634c94

Size

15.3 GB

Last updated

about 2 months ago

docker pull eagleoutice/cran-pkg-db-neo4j