The 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP 2021) invites the submission of long and short papers on substantial, original, and unpublished research in empirical methods for Natural Language Processing. As in recent years, some of the presentations at the conference will be for papers accepted by the Transactions of the ACL (TACL) and Computational Linguistics (CL) journals.
*Important Dates*Anonymity period begins: April 17, 2021 Abstract submission deadline (long & short papers): May 10, 2021 Full paper submission deadline (long & short papers): May 17, 2021 Author response period: July 11-17, 2021 Notification of acceptance (long & short papers): August 25, 2021 Camera-ready papers due (long & short papers): September 9, 2021 Conference: November 7-9, 2021 Workshops & Tutorials: November 10-11, 2021
*All deadlines are 11.59 pm UTC -12h ("anywhere on Earth").*
*Mandatory abstract submission*The paper title, author names, contact details, and a brief abstract must be submitted electronically through the EMNLP 2021 paper submission site by the abstract submission deadline. It will be possible to make minor edits to the title and abstract until the full paper submission deadline but you cannot change authors and subject areas. Submissions with “placeholder” abstracts will be removed without consideration; there is no need to withdraw the submission or notify us.
*Important: if you miss the abstract submission deadline, then you cannot submit the full paper.*
*Submissions*EMNLP 2021 has the goal of a broad technical program. Relevant topics for the conference include, but are not limited to, the following areas (in alphabetical order):
Computational Social Science and Cultural Analytic Dialogue and Interactive Systems Discourse and Pragmatics Efficient Methods for NLP Ethics and NLP Generation Information Extraction Information Retrieval and Text Mining Interpretability and Analysis of Models for NLP Linguistic Theories, Cognitive Modeling and Psycholinguistics Machine Learning for NLP Machine Translation and Multilinguality NLP Applications Phonology, Morphology and Word Segmentation Question Answering Resources and Evaluation Semantics: Lexical, Sentence level, Textual Inference and Other areas Sentiment Analysis, Stylistic Analysis, and Argument Mining Speech, Vision, Robotics, Multimodal Grounding Summarization Syntax: Tagging, Chunking and Parsing
*Paper Submission Information*Long Papers Long paper submissions must describe substantial, original, completed and unpublished work. Wherever appropriate, concrete evaluation and analysis should be included. Review forms will be made available prior to the deadlines. Long papers may consist of up to 8 pages of content, plus unlimited pages for references; final versions of long papers will be given one additional page of content (up to 9 pages) so that reviewers’ comments can be taken into account.
Long papers will be presented orally or as posters as determined by the program committee. The decisions as to which papers will be presented orally and which as poster presentations will be based on the nature rather than the quality of the work. There will be no distinction in the proceedings between long papers presented orally and as posters.
Short Papers Short paper submissions must describe original and unpublished work. Please note that a short paper is not a shortened long paper. Instead short papers should have a point that can be made in a few pages. Some kinds of short papers are:
A small, focused contribution A negative result An opinion piece An interesting application nugget
Short papers may consist of up to 4 pages of content, plus unlimited references. Upon acceptance, short papers will be given 5 content pages in the proceedings. Authors are encouraged to use this additional page to address reviewers’ comments in their final versions.
Short papers will be presented orally or as posters as determined by the program committee. While short papers will be distinguished from long papers in the proceedings, there will be no distinction in the proceedings between short papers presented orally and as posters.
Authorship The author list for submissions should include all (and only) individuals who made substantial contributions to the work presented. Each author listed on a submission to EMNLP 2021 will be notified of submissions, revisions and the final decision. No changes to the order or composition of authorship may be made to submissions to EMNLP 2021 after the abstract submission deadline.
Citation and Comparison You are expected to cite all refereed publications relevant to your submission, but you may be excused for not knowing about all unpublished work (especially work that has been recently posted and/or is not widely cited).
In cases where a preprint has been superseded by a refereed publication, the refereed publication should be cited instead of the preprint version.
Papers (whether refereed or not) appearing less than 3 months before the submission deadline are considered contemporaneous to your submission, and you are therefore not obliged to make detailed comparisons that require additional experimentation and/or in-depth analysis.
For more information, see the ACL Policies for Submission, Review, and Citation <https://www.aclweb.org/portal/content/new-policies-submission-review-and-citation>
Multiple Submission Policy EMNLP 2021 will not consider any paper that is under review in a journal or another conference at the time of submission, and submitted papers must not be submitted elsewhere during the EMNLP 2021 review period. This policy covers all refereed and archival conferences and workshops (e.g., NeurIPS, ACL workshops).
In addition, we will not consider any paper that overlaps significantly in content or results with papers that will be (or have been) published elsewhere. Authors submitting more than one paper to EMNLP 2021 must ensure that their submissions do not overlap significantly (>25%) with each other in content or results.
Ethics Policy Authors are required to honour the ethical code set out in the ACL Code of Ethics <https://www.aclweb.org/portal/content/acl-code-ethics>. The consideration of the ethical impact of our research, use of data, and potential applications of our work has always been an important consideration, and as artificial intelligence is becoming more mainstream, these issues are increasingly pertinent. We ask that all authors read the code, and ensure that their work is conformant to this code. Where a paper may raise ethical issues, we ask that you include in the paper an explicit discussion of these issues, which will be taken into account in the review process. We reserve the right to reject papers on ethical grounds, where the authors are judged to have operated counter to the code of ethics, or have inadequately addressed legitimate ethical concerns with their work.
Authors will be allowed extra space after the 8th page (4th for short papers) for a broader impact statement or other discussion of ethics. The EMNLP review form will include a section addressing these issues and papers flagged for ethical concerns by reviewers or ACs will be further reviewed by an ethics committee. Note that an ethical considerations section is not required, but papers working with sensitive data or on sensitive tasks that do not discuss these issues will not be accepted. Conversely, the mere inclusion of an ethical considerations section does not guarantee acceptance. In addition to acceptance or rejection, papers may receive a conditional acceptance recommendation. Camera-ready versions of papers designated as conditional accept will be re-reviewed by the ethics committee to determine whether the concerns have been adequately addressed. Please read the Ethics FAQ <https://2021.emnlp.org/call-for-papers/ethics-faq> for more guidance on some problems to look out for and key concerns to consider relative to the code of ethics.
Paper Submission and Templates Submission is electronic, using the Softconf START conference management system <https://www.softconf.com/emnlp2021/papers/>. Both long and short papers must follow the EMNLP 2021 two-column format, using the supplied official style files <https://2021.emnlp.org/call-for-papers/style-and-formatting>. *Please do not modify the style files, nor should you use templates designed for other conferences. Submissions that do not conform to the required styles, including paper size, margin width, and font size restrictions, will be rejected without review.*
Optional Supplementary Materials: Appendices, Software and Data Each EMNLP 2021 submission can be accompanied by one PDF appendix for the paper, one PDF for prior reviews and author response, one .tgz or .zip archive containing software, and one.tgz or .zip archive containing data. EMNLP 2021 encourages the submission of these supplementary materials to improve the reproducibility of results, and to enable authors to provide additional information that does not fit in the paper. For example, anonymised related work (see above), preprocessing decisions, model parameters, feature templates, lengthy proofs or derivations, pseudocode, sample system inputs/outputs, and other details that are necessary for the exact replication of the work described in the paper can be put into the appendix. However, the paper submissions need to remain fully self-contained, as these supplementary materials are completely optional, and reviewers are not even asked to review or download them. If the pseudo-code or derivations or model specifications are an important part of the contribution, or if they are important for the reviewers to assess the technical correctness of the work, they should be a part of the main paper, and not appear in the appendix. Supplementary materials need to be fully anonymized to preserve the double-blind reviewing policy.
*Anonymity Period*The following rules and guidelines are meant to protect the integrity of double-blind review and ensure that submissions are reviewed fairly. The rules make reference to the anonymity period, which runs from 1 month before the submission deadline (starting April 17th, 2021) up to the date when your paper is accepted or rejected (August 25th, 2021). Papers that are withdrawn during this period will no longer be subject to these rules.
- You may not make a non-anonymized version of your paper available online to the general community (for example, via a preprint server) during the anonymity period. Versions of the paper include papers having essentially the same scientific content but possibly differing in minor details (including title and structure) and/or in length.
- If you have posted a non-anonymized version of your paper online before the start of the anonymity period, you may submit an anonymized version to the conference. The submitted version must not refer to the non-anonymized version, and you must inform the program chairs that a non-anonymized version exists.
- You may not update the non-anonymized version during the anonymity period, and we ask you not to advertise it on social media or take other actions that would further compromise double-blind reviewing during the anonymity period.
- You may make an anonymized version of your paper available (for example, on OpenReview), even during the anonymity period.
- Note that, while you are not prohibited from making a non-anonymous version available online before the start of the anonymity period, this does make double-blind reviewing more difficult to maintain, and we therefore encourage you to wait until the end of the anonymity period. Alternatively, you may consider submitting your work to the Computational Linguistics journal, which does not require anonymization and has a track for “short” (i.e., conference-length) papers.
- For arXiv submissions, April 17 11:59pm UTC-12h (anywhere on earth) is the latest time the paper can be uploaded.
*Instructions For Double-Blind Review*As reviewing will be double blind, papers must not include authors’ names and affiliations. Furthermore, self-references or links (such as github) that reveal the author’s identity, e.g., “We previously showed (Smith, 1991) …” must be avoided. Instead, use citations such as “Smith previously showed (Smith, 1991) …” Papers that do not conform to these requirements will be rejected without review.
Papers should not refer, for further detail, to documents that are not available to the reviewers. For example, do not omit or redact important citation information to preserve anonymity. Instead, use third person or named reference to this work, as described above (“Smith showed” rather than “we showed”). If important citations are not available to reviewers (e.g., awaiting publication), these paper/s should be anonymised and included in the appendix. They can then be referenced from the submission without compromising anonymity.
Papers may be accompanied by a resource (software and/or data) described in the paper, but these resources should also be anonymized.
Sticky Reviews (optional) Authors resubmitting a paper that has been rejected from another venue are invited to submit alongside their paper the previous version of the paper, the reviews and an author response, following the procedure introduced at EMNLP last year. This is optional. It is designed to mimic the revise-and-resubmit procedure underlying journals like TACL. We expect that the fact that a paper was rejected from another venue will not necessarily affect the paper’s decision in a negative way, but is likely to be beneficial to authors who believe they have addressed the problems identified, and can argue strongly for how the paper has been improved. The prior reviews will not be seen by reviewers, but be used as part of the EMNLP 2021 decision process, primarily by area chairs and program chairs in review quality control, resolving disagreements between reviewers, and in deciding borderline papers.
*Reproducibility Criteria*During the submission process, authors will be asked to answer the questions from the Reproducibility Checklist. The checklist is intended as a reminder to help the authors improve reproducibility of their papers. The papers are not required to meet all reproducibility criteria listed. However, the answers will be made available to the reviewers. Reviewers will be asked to assess the reproducibility of the work as part of their reviews.
The following is a preliminary checklist we plan to use.
For all reported experimental results: - A clear description of the mathematical setting, algorithm, and/or model. - Submission of a zip file containing source code, with specification of all dependencies, including external libraries, or a link to such resources (while still anonymized) - Description of computing infrastructure used - The average runtime for each model or algorithm (e.g., training, inference, etc.), or estimated energy cost - Number of parameters in each model - Corresponding validation performance for each reported test result - Explanation of evaluation metrics used, with links to code
For all experiments with hyperparameter search: - The exact number of training and evaluation runs - Bounds for each hyperparameter - Hyperparameter configurations for best-performing models - Number of hyperparameter search trials - The method of choosing hyperparameter values (e.g., uniform sampling, manual tuning, etc.) and the criterion used to select among them (e.g., accuracy) - Summary statistics of the results (e.g., mean, variance, error bars, etc.)
For all datasets used: - Relevant details such as languages, and number of examples and label distributions - Details of train/validation/test splits - Explanation of any data that were excluded, and all pre-processing steps - A zip file containing data or link to a downloadable version of the data - For new data collected, a complete description of the data collection process, such as instructions to annotators and methods for quality control.
This list is based on Dodge et al, 2019 <https://arxiv.org/pdf/1909.03004.pdf> and Joelle Pineau's reproducibility checklist <https://www.cs.mcgill.ca/~jpineau/ReproducibilityChecklist.pdf>.
*Presentation Requirement*All accepted papers must be presented at the conference to appear in the proceedings. Authors of papers accepted for presentation at EMNLP 2021 must notify the program chairs by the camera-ready deadline if they wish to withdraw the paper.
At least one author of each accepted paper must register for EMNLP 2021 by the early registration deadline.
*ACL Rolling Review* We are working with ACL Rolling Review <https://aclrollingreview.org/> (ARR) towards running a pilot with EMNLP 2021. The plan is for a small proportion of our papers to come from ARR.
ARR is a new review system for *ACL conferences, where reviewing and acceptance of papers to publication venues is done in a two-step process: (1) centralized rolling review and (2) submission to a publication venue. The purpose of ACL Rolling Review is to improve efficiency and turnaround of reviewing in *ACL conferences while keeping the diversity (geographic and otherwise) and editorial freedom.
Cut-off dates for publication and presentation at EMNLP 2021 EMNLP will only consider papers and reviews from ARR if: (1) the papers are submitted to ARR by May 15, 2021 (i.e., the first deadline for ARR), and (2) the reviews are ready by July 29, 2021. Given that this is the first ARR pilot and is running in parallel with the traditional reviewing pipeline, if EMNLP receives a large number of submissions from ARR, we may cap the number of papers that will be evaluated by EMNLP (e.g., 100 submissions). -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: text/html Size: 19397 bytes Desc: not available URL: <https://mailman.uib.no/public/corpora/attachments/20210416/2c53869e/attachment.txt>