{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9c415fcd-464c-5bcf-843e-91f5a6e9f40b","task_key":"default--validation--24054","task_revision_id":"1","upstream_id":"24054","short_description":"HellaSwag validation 24054","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to iron a tie [title] Set up the ironing board. [step] Put the board on a flat surface so that it is steady and will not flip over while you are ironing. Make sure that it is clean and dry before setting your tie down.\",\"endings\":\"['[title] Place your iron at a 90-degree angle at least 12 inches (30 cm) away from the four corners of the fabric. [step] Focus the iron on the right corner of the fabric to cover the stitch; the edge where the stitch meets the strip of material running horizontally along the top of the fabric.', 'Always set the ironing board up so that there is an opening in it, too, to catch any debris that will clog a seam when you iron it. [substeps] Consider folding a towel or piece of cloth around the board if possible.', '[substeps] Make sure that the iron is set at the proper temperature for fabric based ironing techniques. Your iron should be set at medium to high range for beginners.', '[title] Put the iron on the appropriate setting. [step] Read the label on the tie to figure out what material it is made out of.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}