tpickett/domain-discovery-tool repository overview

DDT is an interactive system that helps users explore and better understand a domain (or topic) as it is represented on the Web. It achieves this by integrating human insights with machine computation (data mining and machine learning) through visualization. DDT allows a domain expert to visualize and analyze pages returned by a search engine or a crawler, and easily provide feedback about relevance. This feedback, in turn, can be used to address two challenges:

Guide users in the process of domain understanding and help them construct effective queries to be issued to a search engine; and Configure focused crawlers that efficiently search the Web for additional pages on the topic. DDT allows users to quickly select crawling seeds as well as positive and negatives required to create a page classifier for the focus topic.

This starts the elasticsearch and the DDT server:

docker run -i -p 8084:8084 -p 9200:9200 -t tpickett/domain-discovery-tool /ddt/run_demo.sh

To see the app running, go to:

http://localhost:8084/seedcrawler

Alternativaly, you can also specify an external ElasticSearch server address using an enviroment variable:

docker run -p 8084:8084 -e "ELASTICSEARCH_SERVER=http://127.0.0.1:9200" -i -t tpickett/domain-discovery-tool

⁠Further Documentation

Detailed Description of the tool⁠

Demo Scripts and Videos⁠