Open Datasets

11/01/2025

is it really 1K or just some FB glitch? We are lazy and make around ~0.7 posts per year. Should we (i.e. me) post more often? What kind of data interests you?

10/01/2025

TRELLIS-500K: curated 3D dataset for generation tasks, sourced from Objaverse(XL), ABO, 3D-FUTURE, and others. 500k assets, filtered by aesthetic scores

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

01/04/2022

LAION-5B: a dataset of 5,85 billion CLIP-filtered image-text pairs https://laion.ai/laion-5b-a-new-era-of-open-large-scale-multi-modal-datasets/

31/08/2021

SHIFT15M: Multi-objective Large-Scale Fashion Dataset with Distributional Shifts (15 million fashion images)

Paper: https://arxiv.org/abs/2108.12992
Download: https://github.com/st-tech/zozo-shift15m

21/07/2021

WikiGraphs: A Wikipedia Text - Knowledge Graph Paired Dataset

Paper: https://arxiv.org/abs/2107.09556
GitHub: https://github.com/deepmind/deepmind-research/tree/master/wikigraphs

21/07/2021

AGENT: A Benchmark for Core Psychological Reasoning

Paper: https://arxiv.org/abs/2102.12321
Data: https://www.tshu.io/AGENT/

16/07/2021

Shifts: A Dataset of Real Distributional Shift Across Multiple Large-Scale Tasks

Paper:https://arxiv.org/pdf/2107.07455.pdf
Data: https://github.com/yandex-research/shifts

18/05/2021

Project CodeNet: ~14M code samples, 500M lines of code, 55+ programming languages https://research.ibm.com/blog/codenet-ai-for-code

Project CodeNet is a large dataset aimed at teaching AI to code.

11/12/2020

The State of Open Data 2020 https://digitalscience.figshare.com/articles/report/The_State_of_Open_Data_2020/13227875

02/12/2020

PMLB v1.0: an open source dataset collection for benchmarking machine learning methods (with Python and R interfaces)

Homepage: https://epistasislab.github.io/pmlb/index.html
Paper:https://arxiv.org/pdf/2012.00058.pdf

30/11/2020

SLURP dataset: collection of ~72k audio recordings of user interactions with a home assistant, annotated with 3 levels of semantics: Scenario, Action and Entities (6GB)

Github: https://github.com/pswietojanski/slurp
Paper:https://arxiv.org/pdf/2011.13205.pdf

26/11/2020

Data on COVID-19 by "Our World in Data". Updated daily and includes data on confirmed cases, deaths, hospitalizations, and testing, as well as other variables of potential interest

Data on COVID-19 (coronavirus) cases, deaths, hospitalizations, tests • All countries • Updated daily by Our World in Data - owid/covid-19-data