{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"472393e4-14dd-549c-b6e6-3a8fd63ed5a2","task_key":"default--validation--18033","task_revision_id":"1","upstream_id":"18033","short_description":"HellaSwag validation 18033","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to make an infinity scarf [title] Cut the fabric. [step] Cut a strip of fabric 25 \\\"/64 cm wide, across the full 60 \\\"/150 cm width. [substeps] It's a good idea to prewash the fabric before sewing.\",\"endings\":\"['That way, any possible shrinkage or color run will be dealt with. If preferred, simply measure out one large rectangle from the fabric, in measurements of your choice.', '[title] Fold the square fibers so that they create a v shape. [title] Stitch the ends of the folded corners together to form a v shape.', 'This will ensure that it is finished at the proper length and thickness. Most drugstores need new squares of fabric for unraveling, so ask around before trying.', 'This helps along the seam and prevents it from becoming patchy. Cut the two smaller strips of fabric together first.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}