Harvard Data Science

Thoughts on statistical computing and visualization

All final projects

Diffeo · Scoring model

Document relevancy at scale

An algorithmic relevancy score that beat the previous year’s DARPA challenge winners on a 9 TB text collection.

Diffeo’s result was a relevancy ranker over 9 TB that beat the prior DARPA winners. This desk is a toy corpus that shows the same contest: a body-aware score versus a title-only baseline.

On a fixed five-query challenge, the new score wins where the baseline misses on 0 of 5.

  1. #1

    Coffee shipment windows

    new 5.4 · baseline 6.0

  2. #2

    Port congestion and vessel dwell time

    new 0.0 · baseline 0.0

  3. #3

    Retail demand signals from loyalty cards

    new 0.0 · baseline 0.0

  4. #4

    Carbon accounting for short-haul flights

    new 0.0 · baseline 0.0

  5. #5

    Cross-border bank exposures

    new 0.0 · baseline 0.0

  6. #6

    Highway segment crash narratives

    new 0.0 · baseline 0.0

  7. #7

    A memoir of missed trains

    new 0.0 · baseline 0.0

  8. #8

    Kitchen herbs in winter

    new 0.0 · baseline 0.0

Next: Starbucks — Shipment occurrence and timing