[Corpora-List] Official Launch + Call for Participation: COVIDSearch

Kirk Roberts kirkroberts at gmail.com
Wed Apr 15 19:58:49 CEST 2020

(original Call for Participation below)

TREC-COVID is now officially live, with topics, deadlines, and lots of details. I'll refer you to both the official press release from NIST and OSTP: https://www.nist.gov/news-events/news/2020/04/nist-and-ostp-launch-effort-improve-search-engines-covid-19-research

...and the official website is here: https://ir.nist.gov/covidSubmit/

First deadline: April 23 at 7am.

On Tue, Mar 24, 2020 at 3:55 PM Kirk Roberts <kirkroberts at gmail.com> wrote:

> Researchers, clinicians, and policy makers involved with the response to
> COVID-19 are constantly searching for reliable information on the virus and
> its impact. This presents a unique opportunity for the information
> retrieval (IR) and text processing communities to contribute to the
> response to this pandemic, as well as to study methods for quickly standing
> up such systems for similar future events.
> Recently, the Allen Institute for AI <https://allenai.org/> and
> collaborators announced the availability of an open dataset, the COVID-19
> Open Research Dataset (CORD-19)
> <https://pages.semanticscholar.org/coronavirus-research>. This collection
> of biomedical literature articles currently contains over 40,000 articles
> and will be updated weekly.
> We are announcing an IR challenge for search engines that find relevant
> COVID-related articles within this collection. This challenge will provide:
> - A benchmark set of important COVID-related queries (e.g.,
> "coronavirus risk factors", "COVID-19 ibuprofen")
> - A set of manual judgments for CORD-19 articles on these queries
> - An ongoing leaderboard for comparison of IR systems
> The challenge may in the future expand to more detailed tasks such as
> information-filtering, question-answering, fact-checking, and argument
> mining.
> The current plan is to run the competition in weekly batches, where that
> week's snapshot of CORD-19 is used as the corpus and the results of systems
> participating in that batch are pooled for manual assessment. The task will
> follow the "Cranfield" evaluation procedures that are used in the Text
> Retrieval Conference (TREC) <https://trec.nist.gov/> and related
> challenge evaluations.
> One of the ways we will build topics for the test collection will be to
> solicit them by crowd-sourcing on Twitter. Please reply to our tweets using
> the hashtag, *#COVIDSearch*. (We will assess all nominations and
> incorporate those that best fit the task.)
> The goal of this retrieval challenge is both to help develop systems
> capable of identifying relevant information for the current pandemic, but
> also to scientifically study how retrieval methods can be quickly developed
> for such situations in the future.
> Participants in this project include:
> - Ian Soboroff, National Institute for Standards & Technology (NIST)
> - Ellen Voorhees, National Institute for Standards & Technology (NIST)
> - Dina Demner-Fushman, National Library of Medicine
> - William Hersh, Oregon Health & Science University
> - Kirk Roberts, University of Texas Houston Health Science Center
> - Lucy Lu Wang, Allen Institute for AI
> - Kyle Lo, Allen Institute for AI
> - Steven Bedrick, Oregon Health & Science University
> - Aaron Cohen, Oregon Health & Science University
-------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: text/html Size: 8136 bytes Desc: not available URL: <https://mailman.uib.no/public/corpora/attachments/20200415/1dc5cdcf/attachment.txt>

More information about the Corpora mailing list