{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6c7d01c4-05d4-5e4c-bf74-b526973e1d9e","task_key":"default--validation--620","task_revision_id":"1","upstream_id":"620","short_description":"HellaSwag validation 620","config":"default","split":"validation","body":"{\"activity_label\":\"Youth\",\"ctx\":\"[header] How to ignore bullies [title] Avoid giving a bully the opportunity to bully you. [step] Most bullies will look for easy opportunities to victimize someone, or \\\" easy targets \\\". Try to avoid being an easy target by being aware of the bully's movements when you are in the same room with them and focus on keeping to yourself.\",\"endings\":\"[\\\"[substeps] For example, choose a table further away from the bully at lunch, or a seat in class that is not within eyesight of the bully. It's also important to keep an eye on your belongings and avoid giving the bully an opportunity to bully you.\\\", 'Since bullies often target others because they value solitude and comfort, you will not be able to do this well. [substeps] If you find yourself feeling tempted to call up the bully or cry in front of them, do not.', \\\"[substeps] Don't immediately cover up or hide when in class. Hiding in your locker or a study corner can't be very effective.\\\", '[substeps] Walk carefully, keep your head high and look straight ahead of you and avoid eye contact with strangers. Ignore every over-exaggeration they make, no matter how silly it is.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}