-:jdZddlmZddlmZeGddZdeedeeeeffdZ d eeeefdeefd Z y ) zEvaluation results utilities for the `.eval_results/*.yaml` format. See https://huggingface.co/docs/hub/eval-results for more details. Specifications are available at https://github.com/huggingface/hub-docs/blob/main/eval_results.yaml. ) dataclass)AnyceZdZUdZeed<eed<eed<dZedzed<dZedzed<dZ edzed<dZ edzed <dZ edzed <dZ edzed <dZ edzed <dZedzed <ddZy)EvalResultEntrya Evaluation result entry for the `.eval_results/*.yaml` format. Represents evaluation scores stored in model repos that automatically appear on the model page and the benchmark dataset's leaderboard. For the legacy `model-index` format in `README.md`, use [`EvalResult`] instead. See https://huggingface.co/docs/hub/eval-results for more details. Args: dataset_id (`str`): Benchmark dataset ID from the Hub. Example: "cais/hle", "Idavidrein/gpqa". task_id (`str`): Task identifier within the benchmark. Example: "gpqa_diamond". value (`Any`): The metric value. Example: 20.90. dataset_revision (`str`, *optional*): Git SHA of the benchmark dataset. verify_token (`str`, *optional*): A signature that can be used to prove that evaluation is provably auditable and reproducible. date (`str`, *optional*): When the evaluation was run (ISO-8601 datetime). Defaults to git commit time. source_url (`str`, *optional*): Link to the evaluation source (e.g., https://huggingface.co/spaces/SaylorTwift/smollm3-mmlu-pro). Required if `source_name`, `source_user`, or `source_org` is provided. source_name (`str`, *optional*): Display name for the source. Example: "Eval Logs". source_user (`str`, *optional*): HF user name for attribution. Example: "celinah". source_org (`str`, *optional*): HF org name for attribution. Example: "cais". notes (`str`, *optional*): Details about the evaluation setup. Example: "tools", "no-tools", "chain-of-thought". Example: ```python >>> from huggingface_hub import EvalResultEntry >>> # Minimal example with required fields only >>> result = EvalResultEntry( ... dataset_id="Idavidrein/gpqa", ... task_id="gpqa_diamond", ... value=0.412, ... ) >>> # Full example with all fields >>> result = EvalResultEntry( ... dataset_id="cais/hle", ... task_id="default", ... value=20.90, ... dataset_revision="5503434ddd753f426f4b38109466949a1217c2bb", ... verify_token="eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...", ... date="2025-01-15T10:30:00Z", ... source_url="https://huggingface.co/datasets/cais/hle", ... source_name="CAIS HLE", ... source_org="cais", ... notes="no-tools", ... ) ``` dataset_idtask_idvalueNdataset_revision verify_tokendate source_url source_name source_user source_orgnotesc||j|j |j|j t dyy)NzaIf `source_name`, `source_user`, or `source_org` is provided, `source_url` must also be provided.)rrrr ValueError)selfs _/var/www/html/tokenscope/api/venv/lib/python3.12/site-packages/huggingface_hub/_eval_results.py __post_init__zEvalResultEntry.__post_init__UsH    (D,<,<,HDOOLgoo%s &Mh)returnN)__name__ __module__ __qualname____doc__str__annotations__rr r r r rrrrrrrrr s:xO L J#'cDj'#L#*#D#*!Jd !"Kt""Kt"!Jd !E3:rrentriesrcFg}|D]}|j|jd}|j|j|d<||jd}|j|j|d<|j |j |d<|j dd|j i}|j|j|d<|j|j|d<|j|j|d <||d <|j|j|d <|j||S) aConvert a list of [`EvalResultEntry`] objects to a YAML-serializable list of dicts. This produces the format expected in `.eval_results/*.yaml` files. Args: entries (`list[EvalResultEntry]`): List of evaluation result entries to serialize. Returns: `list[dict[str, Any]]`: A list of dictionaries ready to be dumped to YAML. Example: ```python >>> from huggingface_hub import EvalResultEntry, eval_result_entries_to_yaml >>> entries = [ ... EvalResultEntry(dataset_id="cais/hle", task_id="default", value=20.90), ... EvalResultEntry(dataset_id="Idavidrein/gpqa", task_id="gpqa_diamond", value=0.412), ... ] >>> yaml_data = eval_result_entries_to_yaml(entries) >>> yaml_data[0] {'dataset': {'id': 'cais/hle', 'task_id': 'default'}, 'value': 20.9} ``` To upload eval results to the Hub: ```python >>> import yaml >>> from huggingface_hub import upload_file, EvalResultEntry, eval_result_entries_to_yaml >>> entries = [ ... EvalResultEntry(dataset_id="cais/hle", task_id="default", value=20.90), ... ] >>> yaml_content = yaml.dump(eval_result_entries_to_yaml(entries)) >>> upload_file( ... path_or_fileobj=yaml_content.encode(), ... path_in_repo=".eval_results/hle.yaml", ... repo_id="your-username/your-model", ... ) ``` )idrrevision)datasetr verifyTokenr urlnameuserorgsourcer) rrr r r r r rrrrappend)r resultentryr$datar*s reval_result_entries_to_yamlr/^s*RF).)9)9emm"T  ! ! -"'"8"8GJ +2U[[I    )"'"4"4D  :: ! ::DL    '&+U-=-=%>F  ,!&!2!2v  ,!&!2!2v+ % 0 0u #DN ;; "!KKDM d12 Mrr.cg}|D]}|jd|}|jdi}|jdi}t|d|d|d|jd|jd|jd |r|jd nd |r|jd nd |r|jd nd |r|jdnd |jd }|j||S)aParse a list of dicts into [`EvalResultEntry`] objects. This parses the `.eval_results/*.yaml` format. For the legacy `model-index` format, use [`model_index_to_eval_results`] instead. Args: data (`list[dict[str, Any]]`): A list of dictionaries (e.g., parsed from YAML or API response). Returns: `list[EvalResultEntry]`: A list of evaluation result entry objects. Example: ```python >>> from huggingface_hub import parse_eval_result_entries >>> data = [ ... {"dataset": {"id": "cais/hle", "task_id": "default"}, "value": 20.90}, ... {"dataset": {"id": "Idavidrein/gpqa", "task_id": "gpqa_diamond"}, "value": 0.412}, ... ] >>> entries = parse_eval_result_entries(data) >>> entries[0].dataset_id 'cais/hle' >>> entries[0].value 20.9 ``` r.r$r*r"r rr#r%r r&Nr'r(r)r) rr rr r r r rrrr)getrr+)r.r item entry_datar$r*r-s rparse_eval_result_entriesr4s8GXXfd+ ..B/"-t}W%I&$[[4# 6',2vzz%(.4 6*$.4 6*$,2vzz%(..)   u#$ NrN) r dataclassesrtypingrrlistdictrr/r4rrrr9s~ " OO OdCo)>C4SRUXCWCL/Dc3h$8/T/=R/r