LT4CPR @ AACL-IJCNLP 2026
Language Technologies for Crisis Preparedness and Response
November 9, 2026 · Hybrid: Hengqin, Zhuhai, China + Online
LT4CPR Shared Task

System Submission Validator

Check your team submission ZIP before uploading it. Validation is performed locally in your browser; the selected ZIP is not sent to the LT4CPR website.

System output deadline September 10, 2026 23:59 AoE
Submission unit One team ZIP One or more complete systems
Public test cells 117 per system Across six crises

Required ZIP structure

Each team submits one ZIP. Crisis directories appear directly under each system directory; do not add an extra test/ layer.

<team_id>.zip
└── <team_id>/
    ├── submission.json
    └── systems/
        ├── <system_id_1>/
        │   ├── collapse/
        │   ├── damsafety/
        │   ├── heatwave/
        │   ├── indfire/
        │   ├── landslide/
        │   └── tornado/
        └── <system_id_2>/
            └── ...
Correctsystems/my-system/tornado/cell.report.json
Incorrectsystems/my-system/test/tornado/cell.report.json

Each system must be complete. If a team submits three systems, the ZIP contains 3 × 117 = 351 report files.

Example submission.json 
{
  "submission_format_version": "1.0",
  "team": {
    "id": "uw-crisisnlp",
    "name": "UW CrisisNLP"
  },
  "systems": [
    {
      "id": "retrieval-llm",
      "name": "Retrieval + LLM",
      "primary": true
    },
    {
      "id": "zero-shot-llm",
      "name": "Zero-shot LLM",
      "primary": false
    }
  ]
}

Validate your submission

The browser validator checks packaging, manifest metadata, report schema, file coverage, and evidence tweet IDs.

Download CLI validator
Local validation

This page does not upload or retain your ZIP. A SHA-256 hash is calculated locally for your validation receipt. Official submission happens separately through the LT4CPR Google Form.

✓

Ready to validate

Select your team ZIP to begin.

Final system submission

A successful validation does not itself submit your system. After the archive passes, upload the exact same ZIP through the official LT4CPR Google Form.

Keep the validation receipt and SHA-256. Organizers will rerun the authoritative Python validator on the ZIP collected through Google Forms before evaluation.

Google Form locked Validate a ZIP successfully first.
Continue to LT4CPR Google Form →

Google Forms file upload requires respondents to sign in to a Google Account. The uploaded ZIP is stored in the organizers' Google Drive.

What the validator checks

ArchiveSafe ZIP paths, one team directory, no obsolete test/ layer.
ManifestTeam/system IDs, unique systems, exactly one primary run.
Coverage117 reports per system with the expected crisis-level counts.
Report schemaSchema v1.2, Sections 1–11, valid bullet hierarchy and confidence values.
Evidence IDsEvery cited tweet ID exists in its corresponding test input when the embedded manifest is installed.
Internal leakageObvious construction-time identifiers such as SYN_... are rejected.
Semantic quality is separate. Passing this validator means the submission is structurally valid; it does not mean every generated statement is supported by its cited tweets or matches the reference report.

Questions?

For shared-task submission questions, contact lt4cpr.sharedtask@gmail.com.