Intelligence Task Ontology and Knowledge Graph (ITO)

The Intelligence Task Ontology and Knowledge Graph (ITO) provides a comprehensive, curated model of artificial intelligence tasks, benchmarks and benchmark results, including the biomedical domain.

Research in artificial intelligence (AI) is addressing a growing number of tasks through a rapidly growing number of models and methodologies. This makes it difficult to keep track of where novel AI methods are successfully – or still unsuccessfully – applied, how progress is measured, how different advances might synergize with each other, and how future research should be prioritized. To help address these issues, we created the Intelligence Task Ontology and Knowledge Graph (ITO), a comprehensive, richly structured and manually curated resource on artificial intelligence tasks, benchmark results and performance metrics. The current version of ITO contain 685,560 edges, 1,100 classes representing AI processes and 1,995 properties representing performance metrics. The goal of ITO is to enable precise and network-based analyses of the global landscape of AI tasks and capabilities. ITO is based on technologies that allow for easy integration and enrichment with external data, automated inference and continuous, collaborative expert curation of underlying ontological models. We make the ITO dataset and a collection of Jupyter notebooks utilising ITO openly available.

Processes Sunburst

Click on a segment to zoom in on a sub-process, click on the center of the sunburst to return to its parent. Hover over class to see the number of children.

Data Sunburst

Click on a segment to zoom in on a sub-process, click on the center of the sunburst to return to its parent. Hover over class to see the number of children.

Benchmarks

Natural language processing

Trajectories for average gain ratio

Global SOTA trajectory map for AI tasks in natural language processing. Vertical dashes represent anchors (i.e. first results establishing a new benchmark for a given task). Diamonds represent gains in a SOTA trajectory. Diamond colors represent the ratio of the gain, i.e. darker diamonds represent stronger gains.

Pictorial chart

Computer vision

Trajectories for average gain ratio

Global SOTA trajectory map for AI tasks in natural language processing. Vertical dashes represent anchors (i.e. first results establishing a new benchmark for a given task). Diamonds represent gains in a SOTA trajectory. Diamond colors represent the ratio of the gain, i.e. darker diamonds represent stronger gains.

Pictorial chart