Data Attribution Methods Leaderboards

Survey and ranking of data attribution methods on data selection and downstream application tasks for the Date-LM Evaluation paper.

Leaderboard Submission:

  • To submit your team's scores, click on the "Submit Scores" tab.

Data Attribution Method Categories:

  • Gradient (ex. GradDot, GradSim, LESS, DataInf, EKFAC)
  • Similarity (ex. RepSim)
  • Modeling (ex. MATES)
  • Lexical (ex. BM25)
  • Baseline (ex. GradSafe, OpenAI Moderation, LLM Classifiers)
  • Other

Search Feature:

  • Input the name of the method you would like to search / filter for, and then press "Enter". The original row from the leaderboard table will be displayed.

DATE-LM Task Description: Trained pythia-1B model on Fineweb using Lambada reference dataset. Testing results conducted on 10K step checkpoint.

Ranking Metric: highest score in avg column
{
  • "headers": [
    • "Rank",
    • "Method",
    • "Attribution Method Type",
    • "Model",
    • "Model Size",
    • "avg",
    • "sciq",
    • "arc_easy",
    • "arc_challenge",
    • "logiqa",
    • "boolq",
    • "hellaswag",
    • "piqa",
    • "winogrande",
    • "openbookqa",
    • "Paper/Code/Contact Link"
    ],
  • "data": [
    • [
      • 1,
      • "Rep Sim",
      • "Similarity",
      • "Pythia-1b",
      • "1B",
      • 46,
      • 0.691,
      • 0.441,
      • 0.237,
      • 0.275,
      • 0.561,
      • 0.409,
      • 0.695,
      • 0.537,
      • 0.294,
      • ""
      ],
    • [
      • 2,
      • "Grad Sim",
      • "Gradient",
      • "Pythia-1b",
      • "1B",
      • 45.98,
      • 0.689,
      • 0.44,
      • 0.24,
      • 0.272,
      • 0.556,
      • 0.406,
      • 0.69,
      • 0.537,
      • 0.308,
      • ""
      ],
    • [
      • 3,
      • "Edu",
      • "Other",
      • "Pythia-1b",
      • "1B",
      • 45.83,
      • 0.688,
      • 0.452,
      • 0.24,
      • 0.264,
      • 0.571,
      • 0.409,
      • 0.689,
      • 0.52,
      • 0.292,
      • ""
      ],
    • [
      • 4,
      • "Mates",
      • "Modeling",
      • "Pythia-1b",
      • "1B",
      • 45.76,
      • 0.685,
      • 0.441,
      • 0.241,
      • 0.269,
      • 0.563,
      • 0.408,
      • 0.696,
      • 0.523,
      • 0.292,
      • ""
      ],
    • [
      • 5,
      • "BM25",
      • "Lexical",
      • "Pythia-1b",
      • "1B",
      • 45.72,
      • 0.692,
      • 0.439,
      • 0.239,
      • 0.26,
      • 0.556,
      • 0.406,
      • 0.696,
      • 0.531,
      • 0.296,
      • ""
      ],
    • [
      • 6,
      • "Random",
      • "Other",
      • "Pythia-1b",
      • "1B",
      • 45.34,
      • 0.689,
      • 0.431,
      • 0.244,
      • 0.275,
      • 0.52,
      • 0.407,
      • 0.69,
      • 0.535,
      • 0.29,
      • ""
      ]
    ],
  • "metadata": null
}