ir_datasets provides a common interface to many Information Retrieval (IR) ad-hoc ranking benchmarks, training datasets, and more. It handles downloading, extracting, and providing a unified iterator format for various IR datasets. The library is actively maintained with frequent updates, currently at version 0.5.11, with new datasets and bug fixes released regularly.
pip install ir-datasetsVerified import paths — ran on the pinned version, not inferred.
This quickstart demonstrates how to load a dataset, such as MS MARCO, and iterate through its documents, queries, and relevance judgments using the `ir_datasets.load()` function and its iterator methods.
Prefer `dataset.docs_iter()`, `dataset.queries_iter()`, and `dataset.qrels_iter()` for memory-efficient processing of large datasets. Only use dictionary-based access like `dataset.qrels_dict()` when you are certain the data will fit in memory.
For stable code, rely on the official Python API methods like `dataset.docs_iter()` as described in the main documentation. If using the Beta API, be aware of its experimental nature and potential for changes.
Always check the specific dataset's documentation or the output from `ir_datasets` when attempting to load a new collection. Be prepared to follow manual instructions for non-public datasets.
Install the library using `pip install ir-datasets` or activate the correct virtual environment where it is installed.
Verify the correct dataset ID from the `ir_datasets` documentation (e.g., `ir_datasets.list_datasets()`) and ensure it is spelled correctly.
Clear the `ir_datasets` cache (e.g., by deleting the `~/.ir_datasets/` directory) and retry the dataset access. Ensure sufficient disk space and stable network connectivity, and verify the `IR_DATASETS_HOME` environment variable if using a custom data directory.
Install the necessary optional dependency. For example, if 'warc' is missing, run `pip install ir-datasets[warc]`. Consult the `ir-datasets` documentation for the correct optional dependencies for your dataset.