-:j 4UddlZddlZddlZddlmZddlmZmZddlm Z ddl m Z ddl m Z mZddlmZddlmZdd lmZmZdd lmZmZmZdd lmZmZddlZddlZdd lmZm Z m!Z!dd l"m#Z#ddl$m%Z%ddl&m'Z'ddl(m)Z)m*Z*ddl+m,Z,m-Z-m.Z.m/Z/m0Z0ddl1m2Z2m3Z3ddl4m5Z5m6Z6m7Z7m8Z8m9Z9m:Z:m;Z;ddlZ>m?Z?m@Z@mAZAddlBmCZCe GddZDe GddeDZEe GddeDZFeGejZIeeJd<GddeIZKGd d!ejeK"ZLGd#d$ejjZOGd%d&ejjZPd'eQd(eQfd)ZRd*eQd(eQfd+ZSd,eQd-eTdzd(efd.ZUd/eLd,eQd0eQd1eVd2eQf d3ZWd4ZXeLZYy)5N)deque)IterableIterator) ExitStack)deepcopy) dataclassfield)datetime)chain)Path PurePosixPath)AnyNoReturnUnion)quoteunquote)_DEFAULT_CALLBACK NoOpCallback TqdmCallback) apply_config) isfilelike) constants)CommitOperationCopyCommitOperationDelete)BucketNotFoundErrorEntryNotFoundErrorHfHubHTTPErrorRepositoryNotFoundErrorRevisionNotFoundError) hf_hub_urlhttp_get)SPECIAL_REFS_REVISION_REGEX BucketFile BucketFolderHfApiLastCommitInfoRepoFile RepoFolder)HFValidationErrorhf_raise_for_status http_backoffhttp_stream_backoff parse_hf_uri)md5c2eZdZUdZeed<eed<defdZy)HfFileSystemResolvedPathz_Top level Data structure containing information about a resolved Hugging Face file system path.rootpathreturncV|jd|jjdS)N/)r2r3rstripselfs `/var/www/html/tokenscope/api/venv/lib/python3.12/site-packages/huggingface_hub/hf_file_system.py unresolvez"HfFileSystemResolvedPath.unresolve+s&))Adii[)0055N)__name__ __module__ __qualname____doc__str__annotations__r;r<r:r1r1$si I I636r<r1ceZdZUdZeed<eed<eed<eed<edZeed<edZeed <ed d Z ed zed <d Z y )"HfFileSystemResolvedRepositoryPathzLData structure containing information about a resolved path in a repository. repo_typerepo_idrevision path_in_repoFinitr2r3N)defaultrepr _raw_revisioncbtjj|jd|jz}|j r|d|j |_nD|jtjk7r |dt|j|_n||_|j|_ y)N@) rREPO_TYPES_URL_PREFIXESgetrFrGrNr2rHDEFAULT_REVISION safe_revisionrIr3)r9 repo_paths r: __post_init__z0HfFileSystemResolvedRepositoryPath.__post_init__=s5599$.."MPTP\P\\   $+Qt'9'9&:;DI ]]i88 8$+Q}T]]'C&DEDI!DI%% r<) r=r>r?r@rArBr r2r3rNrWrCr<r:rErE/sTVN LM5!D#!5!D#!!&d ?M3:?&r<rEc>eZdZUdZeed<edZeed<dZy)HfFileSystemResolvedBucketPathzHData structure containing information about a resolved path in a bucket. bucket_idFrJr2c,d|jz|_y)Nzbuckets/)rZr2r8s r:rWz,HfFileSystemResolvedBucketPath.__post_init__Os/ r<N) r=r>r?r@rArBr r2rWrCr<r:rYrYHsRN5!D#!0r<rY _cached_basec(eZdZdZfdZdZxZS)_Cacheda Metaclass for caching HfFileSystem instances according to the args. This creates an additional reference to the filesystem, which prevents the filesystem from being garbage collected when all *user* references go away. A call to the :meth:`AbstractFileSystem.clear_instance_cache` must *also* be made for a filesystem instance to be garbage collected. This is a slightly modified version of `fsspec.spec._Cached` to improve it. In particular in `_tokenize` the pid isn't taken into account for the `fs_token` used to identify cached instances. The `fs_token` logic is also robust to defaults values and the order of the args. Finally new instances reuse the states from sister instances in the main thread. c2t||i|i|_yN)super__init___cache)clsargskwargs __class__s r:rbz_Cached.__init__gs $)&) r<ct||}|jdd}|j|tjg|i|}|j|tj j g|i|}|s0|jr$||jvr||_ |j|Stj|g|i|}|s\|jrP||jvrB|j|j}|jD]\}} t||| ||_||_||_|jr|s||_ ||j|<|S)Nskip_instance_cacheF)rpop _tokenize threading get_ident main_threadidentcachablerc_latesttype__call___get_instance_stateitemssetattr _fs_token_ storage_argsstorage_options) rdrerfskipfs_tokenfs_token_main_threadobjinstance_stateattr state_values r:rsz_Cached.__call__osHc6*zz/7 3==i&9&9&;MdMfM,s}}S)2G2G2I2O2OaRVaZ`a SZZ)?"CK::h' '--5d5f5CCLL-ASZZ-O!$,@!A!U!U!W)7)=)=)?4%D+C{34%CN#C "(C ||D& '* 8$Jr<)r=r>r?r@rbrs __classcell__rgs@r:r^r^Ws r<r^ceZdZdZdZdZddddddedzdeezdzdedzd edzffd Z e d ed efd Z dedededzd e ee dzffdZded e ee dzffdZ d6dededzd eezfdZd6dedzd dfdZ d7dedededzdedzd edf dZd6dededzd dfdZ d8dedededzdedzd df dZ d9dededededzd eeeeefzf d Z d:dededededzd edzdedzf d!Zded"eded eeezfd#Z ded e!e eeeeefffd$ Z"d6dededzd eeffd% Z# d;dededzd&ededededzd eeeeeeeffzfd'Z$d6d(ed)ededzd dfd*Z%ded e&fd+Z'd HfFileSystemaz Access a remote Hugging Face Hub repository as if were a local file system. > [!WARNING] > [`HfFileSystem`] provides fsspec compatibility, which is useful for libraries that require it (e.g., reading > Hugging Face datasets directly with `pandas`). However, it introduces additional overhead due to this compatibility > layer. For better performance and reliability, it's recommended to use `HfApi` methods when possible. The file system supports paths for the `hf://` protocol, which follows those URL schemes: * Models, Datasets and Spaces repositories: ``` hf://[@]/ hf://datasets/[@]/ hf://spaces/[@]/ ``` * Buckets (generic storage): ``` hf://buckets// ``` Note: when using the [`HfFileSystem`] directly, passing the `hf://` protocol prefix is optional in paths. Args: endpoint (`str`, *optional*): Endpoint of the Hub. Defaults to . token (`bool` or `str`, *optional*): A valid user access token (string). Defaults to the locally saved token, which is the recommended method for authentication (see https://huggingface.co/docs/huggingface_hub/quick-start#authentication). To disable authentication, pass `False`. block_size (`int`, *optional*): Block size for reading and writing files. expand_info (`bool`, *optional*): Whether to expand the information of the files. **storage_options (`dict`, *optional*): Additional options for the filesystem. See [fsspec documentation](https://filesystem-spec.readthedocs.io/en/latest/api.html#fsspec.spec.AbstractFileSystem.__init__). Usage: ```python >>> from huggingface_hub import hffs >>> # List files >>> hffs.glob("my-username/my-model/*.bin") ['my-username/my-model/pytorch_model.bin'] >>> hffs.ls("datasets/my-username/my-dataset", detail=False) ['datasets/my-username/my-dataset/.gitattributes', 'datasets/my-username/my-dataset/README.md', 'datasets/my-username/my-dataset/data.json'] >>> # Read/write files >>> with hffs.open("my-username/my-model/pytorch_model.bin") as f: ... data = f.read() >>> with hffs.open("my-username/my-model/pytorch_model.bin", "wb") as f: ... f.write(data) ``` Specify a token for authentication: ```python >>> from huggingface_hub import HfFileSystem >>> hffs = HfFileSystem(token=token) ``` rPhfN)endpointtoken block_size expand_inforrrrct||i||xstj|_||_t |||_||_||_ i|_ i|_ i|_ y)N)rr) rarbrENDPOINTrrr&_apirr_repo_and_revision_exists_cache_bucket_exists_cachedircache)r9rrrrreryrgs r:rbzHfFileSystem.__init__sj $2/2 6I$6$6  859 $&rt,NP!:< r<threading_identr4c(|jdxstj|d<|jd|d<t|Dcic]}||| }}||||f}t t |j }|jScc}w)zDeterministic token for cachingrr)rSrrsortedr/rAencode hexdigest)rdrrerfkey tokenize_argshs r:rkzHfFileSystem._tokenizes$ZZ 3Iy7I7Iz **W-w.4Vn=s#vc{"==otV< M"))+ ,{{} >s BrFrGrHc|||f|jvrR |jj|||tjd|j|||f<d|j||df<|j|||fS#t t f$r2}d|f|j|||f<d|f|j||df<Yd}~Nd}~wt$r0}d|f|j|||f<d|j||df<Yd}~d}~wwxYw)N)rHrFtimeoutTNF)rr repo_inforHF_HUB_ETAG_TIMEOUTrr*r )r9rFrGrHes r:_repo_and_revision_existz%HfFileSystem._repo_and_revision_exists! w )1U1U U ^ ##h)YMjMj$Xb44i(5STS]44i$5OP33Y4RSS,->? \W\^_W_44i(5STSXZ[S[44i$5OP( ^W\^_W_44i(5STS]44i$5OP ^s#-A66C.(B22 C.>&C))C.rZc||jvr+ |jj|d|j|<|j|S#t$r}d|f|j|<Yd}~.d}~wwxYw)NrF)rr bucket_infor)r9rZrs r:_bucket_existszHfFileSystem._bucket_exists st D55 5 B %%i08B)))4((33 ' @7?   !%!4!4VYY!? M3 %dC01FIIFL_L_` `  FOO$?FOOW_D_ :>C9KyOmOm9m 3 21 5sw  1 7 7Q ? GXGJhX^^C-C]egi\j * l'./?'@ $'83 34L3MMmnvmwxLM,0+H+HV]_g+h ( #S*%dC05 Wh L\  ?? &8+;H'+'D'DV[[RXR[R[]e'f$& !$ ,$( $;6??69=C9KyOmOm9m 3 21 5sw ,223:1=LNTNaNa<(IC0C0C,Dq,H*IJgsL'389S9S1 KKHf.A.AQ]  r<c|s5|jj|jjy|j|}|j }|r0|jj |d|j |}|r0|jst|trq|jj |j|jdfd|jj |j|j|jfdy|jj |jdyy)ac Clear the cache for a given path. For more details, refer to [fsspec documentation](https://filesystem-spec.readthedocs.io/en/latest/api.html#fsspec.spec.AbstractFileSystem.invalidate_cache). Args: path (`str`, *optional*): Path to clear from cache. If not provided, clear the entire cache. N)rclearrrr;rj_parentr3 isinstancerErFrGrHrrZ)r9r3 resolved_paths r:invalidate_cachezHfFileSystem.invalidate_cachecs  MM   !  0 0 6 6 8 --d3M **,D !!$-||D) !%%m-OP88<<&00-2G2GNPT88<<&00-2G2GI_I_`bf--11-2I2I4P&r<mode)HfFileSystemFileHfFileSystemStreamFilec ||n |j}|||d<d|vr td|dk(rt||f||d|St||f||d|S)Nraz/Appending to remote files is not yet supported.r)rrH)rrrr)r9r3rrrHrfs r:_openzHfFileSystem._opensr$.#9Zt  !#-F< $;%&WX X ?)$]4(]V\] ]#D$WTHWPVW Wr<c |j||}t|tr3|jj |j |j gnr|jj|j|j|j|j|j|jd|jd|j|jy)NrHdeletecommit_messagecommit_description)rIrGrrFrHrrr3)rrrYrbatch_bucket_filesrZr3 delete_filerIrGrrFrHrSrr;)r9r3rHrfrs r:_rmzHfFileSystem._rms))$)B m%C D II ( ()@)@-J\J\I] ( ^ II ! !*77%--jj'11&//%zz*:;#)::.B#C "  =#:#:#<=r< recursivemaxdepthc F|j||}|j||||}t|trb|Dcgc]/}|j |r|j|j 1}}|j j|j|n|Dcgc]/}|j |r|j|j 1} }| D cgc]} t| } } dd} | |rdndz } | |d |dndz } |j j|j|j|j| |j|jd | |jd  |j!|j#y cc}wcc}wcc} w)a Delete files from a repository. For more details, refer to [fsspec documentation](https://filesystem-spec.readthedocs.io/en/latest/api.html#fsspec.spec.AbstractFileSystem.rm). > [!WARNING] > Note: When possible, use `HfApi.delete_file()` for better performance. Args: path (`str`): Path to delete. recursive (`bool`, *optional*): If True, delete directory and all its contents. Defaults to False. maxdepth (`int`, *optional*): Maximum number of subdirectories to visit when deleting recursively. revision (`str`, *optional*): The git revision to delete from. r)rrrHr)rIzDelete  z recursively rPNz up to depth rr)rGrFr operationsrHrrr)r expand_pathrrYisdirr3rrrZr create_commitrGrFrrHrSrr;) r9r3rrrHrfrpathsr paths_in_reporIrrs r:rmzHfFileSystem.rms6))$)B   XX` a m%C D?D]tDJJW[L\d''-22]F] II ( ()@)@ ( PFKddSWS]S]^bScT..t499dMd_lm|/\JmJm&tfA.N  nr AN H

[!WARNING] > Note: When possible, use `HfApi.list_repo_tree()` for better performance. Args: path (`str`): Path to the directory. detail (`bool`, *optional*): If True, returns a list of dictionaries containing file information. If False, returns a list of file paths. Defaults to True. refresh (`bool`, *optional*): If True, bypass the cache and fetch the latest data. Defaults to False. revision (`str`, *optional*): The git revision to list from. Returns: `list[Union[str, dict[str, Any]]]`: List of file paths (if detail=False) or list of file information dictionaries (if detail=True). r)rrHNnamer)rr;_ls_treerr3rrr) r9r3rrrHrfroutos r:lszHfFileSystem.lss4))$)B &&( 2$--SgSFSCs:c":1V9"::" 2 %%%dD1 #dmmDLL$6eRZe^de%  !7QvY$%6177C73x1}%dD1 2#;sMA C)!C&4%BC& B(%C&'B((C&/ C=C"C&%C&c  ||n|j |jnd}|j||}|j}|j}|r|nd}g} ||jvrh|se|j|} | j | g} |rd} t | D cgc]} | ddk(s | | fc} }|r|j\} }|| |kru|d|jvr| j|dnO|j|d} | j | |j | D cgc]} | ddk(r| dz| fc} |rg}|r7t|tr'| Dcgc]}|d |j|d}}|r| s|rC|r@tjj| |z}|jd s||k(s|t!| |vr|j#d n|j|}|"|t%|dj'd }||z}| Dcgc]}|dj)|d zr|} }t+|jD]3}|j)|d zs|jj-|d5|jj-|d| j |j/||d ||| | St|t0r)|j3|j4|j| }nI|j6j9|j:|j|||j<|j> }|D]c} |d z| jz}t| t@rH|| jBd| jD| jF| jH| jJ| jLd}nt| tNr2|| jBd| jH| jP| jRd}n>t| tTr|dd| jV| jJd}n|dd| jRd}|j|d}|jjY|gj||t%|dj'd } || |ksS| j|f| Scc} wcc} wcc}wcc}w)NFrrrr directoryr last_commitr6T)rrrHrr)prefixr)rexpandrHrFfilersizerrblob_idlfsxet_hashrsecurity)rrrrrmtime uploaded_atrrrrrtree_idr)rrrrr)-rrr;r2rextendrpopleftappendrrErosr3 commonprefixendswithr r7rr startswithlistrjrrY_list_bucket_tree_with_foldersrZrlist_repo_treerGrHrFr(rrrrrrr$rrr)r setdefault)r9r3rrrHrrr root_pathrcached_path_infosdirs_not_in_dircachedepth path_info dirs_to_visitdir_infodirs_not_expandedr common_prefix common_pathcommon_path_depth cached_pathtree cache_pathcache_path_info parent_paths r:rzHfFileSystem._ls_trees'2KTM]M]Mi9I9Iot ))$)B &&(!&& (8a 4==  $ d 3  JJ( )#%  %9JoIiX^N_cnNneY'o! $&3&;&;&=OE8'5H+<#F+4==@0778HI04 hv>N0O-JJ'89)006G!"(1'0'8K'G&+QY $:!"$ !# z-9[\FI$fQ}M]MeT\\!F)%<$f!$f2 HY!# 4 45IL]5] ^ %--c2$ 1$.BDU(VV"((-m4 '(3CIK(@(F(Fs(K% 11H"%UQQvY-A-A+PSBS-TqUU#' #6=K"--kC.?@ ))+t<= !!+t4 MM#"+ $!)$/!) " J s-)GH::!++(--'; yy//!))!&&'&*33+55 0") 0 &_y~~= i2 * ) &#,#4#4(}}$-$6$6'0'<'<$-$6$6 'O :6 * ) &$-$6$6!*'0'<'< 'O :6 * ! +#,#4#4'0'<'< 'O!+ ! +'0'<'< 'O #ll?6+BC  ((b9@@Q"3t9;/55c:#u'8JJ/S) 0T ap!"%g"Vs* R:%R:&R? S)SS 8S rc|jj|||}i}|rd|jdznd}g}|D]}|j||st t |j jd| dz D]} tdt| |j} | j |vr!|j| | || j <S| j`|| j } | j| j| jks| j| _ |std|d |d |S) z8>v>>r<withdirsc *||dkr td|j||}|j} |j|fd||d|} |s| D cgc] } | ddk7s | } } n#|j|fi|} | ddk(r| g| zn| } | D cic]} | d|  } } t| } |s| S| D cic]} | | |  c} Scc} wcc} w#t $r8 |j|fd |i|dd k(r|ii} ni} n#t $ri} YnwxYwYmwxYwcc} w) a List all files below path. For more details, refer to [fsspec documentation](https://filesystem-spec.readthedocs.io/en/latest/api.html#fsspec.spec.AbstractFileSystem.find). Args: path (`str`): Root path to list files from. maxdepth (`int`, *optional*): Maximum depth to descend into subdirectories. withdirs (`bool`, *optional*): Include directory paths in the output. Defaults to False. detail (`bool`, *optional*): If True, returns a dict mapping paths to file information. Defaults to False. refresh (`bool`, *optional*): If True, bypass the cache and fetch the latest data. Defaults to False. revision (`str`, *optional*): The git revision to list from. Returns: `Union[list[str], dict[str, dict[str, Any]]]`: List of paths or dict of file information. rzmaxdepth must be at least 1rT)rrrrrrrrHr)rrr;rinforFileNotFoundErrorr)r9r3rrrrrHrfrrrrnamesrs r:findzHfFileSystem.findsh@  HqL:; ;))$)B &&( .$--cgPXc\bcC"%BQ6k)AqBB&DIId5f5 +4V+< +KykC'QT),-A1V9a<-C-s L056D#d)O6 6C .!"  499T?H??G6Q*CC$   *7sMC  C"C C2 D D "C98D 9 DD DD  D path1path2c 6|j||}|j||}t|tst|tr td|j|jk(xr|j |j k(}|rd|d|}|j j|j |j|j|jd||jddt|j|j|jg n|j|d |j5} | j} d d d d|d|}|j j |j|j |j|j|j|jd||jd |j!|j# |j!|j# y #1swYxYw)a Copy a file within or between repositories. > [!WARNING] > Note: When possible, use `HfApi.upload_file()` for better performance. Args: path1 (`str`): Source path to copy from. path2 (`str`): Destination path to copy to. revision (`str`, *optional*): The git revision to copy from. rz)Copy from/to buckets is not available yetzCopy z to rrrP)src_path_in_reporI src_revision)rGrFrHrrrrbNpath_or_fileobjrIrGrrFrHrrr)rrrYrrFrGrrrHrSrrIopenread upload_filerrr;) r9r#r$rHrfresolved_path1resolved_path2 same_reporfcontents r:cp_filezHfFileSystem.cp_files **58*D**58*D n&D E :J &&QR R  $ $(@(@ @ u^E[E[_m_u_uEu  $UG4w7N II # #&..(22'00%zz*:NK#)::.BB#G')7)D)D%3%@%@%3%<%< $ 5$1H1HI #Q&&( #$UG4w7N II ! ! '+88&..jj(22'00%zz*:NK#)::.B#C "  >#;#;#=> >#;#;#=> # #s 9HHc |j|fii|ddi}d|vr#|dtd|d|djSd|vr |dr|dSd|vr |dr|dStd|d |d ) ak Get the last modified time of a file. For more details, refer to [fsspec documentation](https://filesystem-spec.readthedocs.io/en/latest/api.html#fsspec.spec.AbstractFileSystem.modified). Args: path (`str`): Path to the file. Returns: `datetime`: Last modified time of the file. rTrz9'modified' is not implemented for repository paths like 'rrrz'Cannot determined 'modified' for path 'z ' (info: ))rrdate)r9r3rfrs r:modifiedzHfFileSystem.modifiedSstyyA!@F!@M4!@A D M"*),efjekkl*mnn &++ + _g= d "tM': & &%(OPTvU^_c^dde&fg gr<c |j||}|j}|jd|j |jnd}|js|ddd}t |t rd|d<t |t r|r|jj|j|j|j d }i|dt|j|j|j d }nBt |t r^|j#|} |j%| | |j&| D cgc] } | d|k(s | } } | s t)|d| d}nd}|j#|} |s| |j&vr|j%| | |j&vr8|j&| D cgc] } | d|k(s | } } | s t)|d| d}|s| |rS|rP|dJ|jj+|j|j,||j|j} | s t)|d| d} t |j|j|jd|j.j}t | t0rX|dz| jz| j2d| j4| j6| j8| j:| j<d}n,|dz| jzdd| j>| j:d}|sdDcic]}||| }}|J|Scc} wcc} wcc}w)aS Get information about a file or directory. For more details, refer to [fsspec documentation](https://filesystem-spec.readthedocs.io/en/latest/api.html#fsspec.spec.AbstractFileSystem.info). > [!WARNING] > Note: When possible, use `HfApi.get_paths_info()` or `HfApi.repo_info()` for better performance > (or `HfApi.get_bucket_paths_info()` or `HfApi.bucket_info()` for buckets) Args: path (`str`): Path to get info for. refresh (`bool`, *optional*): If True, bypass the cache and fetch the latest data. Defaults to False. revision (`str`, *optional*): The git revision to get info from. Returns: `dict[str, Any]`: Dictionary containing file information (type, size, commit info, etc.). rrNFrr)rrrrr)rFrH)oidtitler6)rr)rr)rrHrFrP)rIrNr6rrr) rr;rSrr3rrErlist_repo_commitsrGrFrHr' commit_idr; created_atrYrrrrget_paths_inforIrNr(rrrrrrr)r9r3rrHrfrrrrr rout1 paths_inforrks r:rzHfFileSystem.infolsX,))$)B &&(jj t/?/?/K4++QV !!#C -)KL%)M"-)KLQ\"ii99!))]5L5LWdWmWm: ##1'119J9JQ\QgQg$ 'E F,,t,K GGKG 1#}}[9O!QvY$=NAODO%dD1q'CC,,t,K;dmm#C $dmm+#'==#=Sa6dARSS)$51g#++##mBTB\!YY55!))!..&*33+55 6 ")$5&qM >!++!))!**!#"/"="=  )+ i2 )C).. @ ) &#,#4#4(}}$-$6$6'0'<'<$-$6$6 C!*C).. @ ! +#,#4#4'0'<'< C#.FG1c!f9GCG qPTTHs MM> M  M 0 Mc |jddr|j||j|fi|y#t$rYywxYw)a Check if a file exists. For more details, refer to [fsspec documentation](https://filesystem-spec.readthedocs.io/en/latest/api.html#fsspec.spec.AbstractFileSystem.exists). > [!WARNING] > Note: When possible, use `HfApi.file_exists()` for better performance. Args: path (`str`): Path to check. Returns: `bool`: True if file exists, False otherwise. rFT)rSrrOSError)r9r3rfs r:existszHfFileSystem.existssK  zz)U+%%d+ DIId %f %  s69 AAcP |j|ddk(S#t$rYywxYw)ai Check if a path is a directory. For more details, refer to [fsspec documentation](https://filesystem-spec.readthedocs.io/en/latest/api.html#fsspec.spec.AbstractFileSystem.isdir). Args: path (`str`): Path to check. Returns: `bool`: True if path is a directory, False otherwise. rrrFrrDr9r3s r:rzHfFileSystem.isdirs1 99T?6*k9 9    %%cP |j|ddk(S#t$rYywxYw)a` Check if a path is a file. For more details, refer to [fsspec documentation](https://filesystem-spec.readthedocs.io/en/latest/api.html#fsspec.spec.AbstractFileSystem.isfile). Args: path (`str`): Path to check. Returns: `bool`: True if path is a file, False otherwise. rrrFrGrHs r:isfilezHfFileSystem.isfiles1 99T?6*f4 4  rIcx|j|}t|tr2|jd|jdt |j }nBt|j|j|j|j|j}|j|r|jddd}|S)z Get the HTTP URL of the given path. Args: path (`str`): Path to get URL for. Returns: `str`: HTTP URL to access the file or directory on the Hub. z /buckets/z /resolve/)rFrHrz/tree/r)rrrYrrZrr3r!rGrIrFrHrreplace)r9r3rurls r:rNzHfFileSystem.urls))$/ m%C D]]O9]-D-D,EYuUbUgUgOhNijC%%**'11&// C ::d ++k8Q7C r<c |jd}t|jdhz }t|tt frt |dkDrt | ||f||d|St|r|}n)|j|rtj|dyt|ttfr4tjtjj!|dd}|t#|d}d}|j%} |j'|| } |j)|| d } |j+|  t-|j/| j1||| d|j2j5t|t r |j6nd |j9| |r|j;yy#|r|j;wwxYw) ad Copy single remote file to local. > [!WARNING] > Note: When possible, use `HfApi.hf_hub_download()` or `HfApi.download_bucket_files` for better performance. Args: rpath (`str`): Remote path to download from. lpath (`str`): Local path to download to. callback (`Callback`, *optional*): Optional callback to track download progress. Defaults to no callback. outfile (`IO`, *optional*): Optional file-like object to write to. If provided, `lpath` is ignored. rHr)callbackoutfileT)exist_okNFwbrr)rN temp_filedisplayed_filename expected_size resume_sizeheaders _tqdm_bar)rSsetkeysrrrrraget_filerrrmakedirsrAr r3dirnamer+tellrrset_sizer"rNr;r_build_hf_headerstqdmseekclose) r9rpathlpathrPrQrfrHunhandled_kwargs close_file initial_posresolve_remote_pathrVrgs r:r\zHfFileSystem.get_file1s$::j)v{{}- <(\<$@ASIYEZ]^E^7#E5_8W_X^_ _ e G ZZ  KK - ec4[ ) KK. > ?5$'GJlln #///I %( ;FC -( HH0::<=!#(+ 335+5h +M(--SW  LL % z s A4GG#ctd)zA context within which files are committed together upon exit Requires the file class to implement `.commit()` and `.discard()` for the normal and exception cases. (Transactional commits are not supported.rr8s r: transactionzHfFileSystem.transactionos""LMMr<ctd)z@Begin write transaction for deferring files, non-context versionrlrmr8s r:start_transactionzHfFileSystem.start_transactionzs""LMMr<cptt||j|j|j ffSr`) make_instancerrrxryrtr8s r: __reduce__zHfFileSystem.__reduce__s9 J      $ $ &    r<ct|jt|jt|jdS)N)rrr)rrrrr8s r:rtz HfFileSystem._get_instance_states4 //78\8\/]$,T-F-F$G  r<r`)r(NN)FNN)TFN)FFNNN)NFFFN)FNr4N)5r=r>r?r@ root_markerprotocolrAboolintrb classmethodrktuple ExceptionrrrErYrrrrrrrdictrrrrr$r%rrrrr"r3r r7rrErrKrNrr\propertyrnrprsrtrrs@r:rrsh@DKH $#'!%#' <*<czD < $J < D[ <2    TT'*T69DjT tY%% &T&44dI>sTz>t>& ## 0>0>0>* 0> * 0> 0>f]a););!%);7;);ORUYz); cDcN" #);\ ##'#KKK K * K D[ K*KZ&&&)&6:& *|+ ,&P77(5d3iQUVYQZAZ;[2\7 ??d ?S ?* $#;7;7*;7 ;7  ;7  ;7*;7 cT#tCH~-. .;7z:?S:?:?d :?X\:?xhShxh2lltlsTzl_cdgildl_ml\2$$4/@< |NNN   r<r) metaclassceZdZddedededzffd ZfdZdeded efd Z dd Z dd e d dfd Z dfd Z d efdZxZS)rNfsr3rHc  |j|||_t |||jj fi||y#t$r)}d|jddvrt|d|d}~wwxYw)NrwrrPB. Make sure the repository and revision exist before writing data.)rrr rSrarbr;)r9rr3rHrfrrgs r:rbzHfFileSystemFile.__init__s !#!ID  T//99;FvF ! fjj,,'c\]   sA A9$A44A9c:t|dsyt| SNr)hasattrra__del__r9rgs r:rzHfFileSystemFile.__del__st_- w  r<startendr4cdd|d|dz i|jjj}|j}t d||t j }t||jS)Nrangebytes=-rGETrXr) rrrarNr,rHF_HUB_DOWNLOAD_TIMEOUTr+r2)r9rrrXrNrs r: _fetch_rangezHfFileSystemFile._fetch_rangesm veWAcAgY/ ggll,,. hhj Wi>_>_ `Ayyr<c<tjdd|_y)Nzhffs-F)rr)tempfileNamedTemporaryFilerTr8s r:_initiate_uploadz!HfFileSystemFile._initiate_uploads!44GERr<finalc J|jjd|jj}|jj ||r|jj t |jtrg|jjj|jj|jj|jjfgn|jjj|jj|jj |jj"|jj$|jj&|jj(|j*j-d|j*j-dt/j0|jj|jj3|jj5yy)Nr)addrrr)r)bufferrcr,rTwriterdrrrYrrrrZrr3r-rIrGrrFrHrfrSrremoverr;)r9rblocks r: _upload_chunkzHfFileSystemFile._upload_chunks    " U#  NN "$,,.LM //&&008K8KTM_M_MdMd7e6f0 (($(NN$7$7!%!3!3!@!@ ..66''--"00::!//88#';;??3C#D'+{{7K'L)  IIdnn)) * GG $ $''113 % % r<cH|jdk(rx||dk(rq|jdk(rb|jj|jdd5}|j }|xjt |z c_|cdddSt||S#1swYxYw)zRead remote file. If `length` is not provided or is -1, the entire file is downloaded and read. On POSIX systems the file is loaded in memory directly. Otherwise, the file is downloaded to a temporary file and read from there. r(Nr9r)r)rlocrr+r3r,rra)r9lengthr1rrgs r:r,zHfFileSystemFile.reads 99 &.FbLdhhRSmdii!< ffhCH$  w|F##   s 0BB!cL|jj|jSr`rrNr3r8s r:rNzHfFileSystemFile.urlww{{499%%r<r`ru)Fr9)r=r>r?rrArbrrybytesrrrxrr,rNrrs@r:rrsj < s cDj ! #CES4D4 $&S&r<rc eZdZ ddededededzdedef fd Zdd ed efd Zdd efd Zdde e d ede fdZ defdZ fdZ dZdZxZS)rNrr3rrHr cache_typec |dk7rtd||dk7rtd|d|vrtd|d |j|||_|jj dd |_t | ||jj f|||d |d|_ |t|_ d|_ t|_y#t$r-}d|j d d vrt|d |Yd}~d}~wwxYw)Nrz:HfFileSystemStreamFile only supports block_size=0 but got nonez?HfFileSystemStreamFile only supports cache_type='none' but got rz;HfFileSystemStreamFile only supports reading but got mode='rrrrPr)rrrrr)rrrr rSr;detailsrarbresponser _exit_stack_stream_iterator bytearray_stream_buffer) r9rr3rrHrrrfrrgs r:rbzHfFileSystemStreamFile.__init__s7 ?YZdYefg g  ^_i^jkl l $;Z[_Z``abc c !#!ID !% 2 2 < < >M   "",,. 59j]g kq 04  $;8<'k! fjj,,'c\]- sC D#DDrwhencecX|dk(r|dk(ry||jk(r|dk(rytd)NrrzCannot seek streaming HF file)rr)r9rrs r:rczHfFileSystemStreamFile.seeks2 !8!  $((?v{ 899r<rc|j|jd} |j |jy|j|j|}|xjt |z c_|S#t $r>|j|jj|r|jd}YnwxYw)zRead the remote file. If the file is already open, we reuse the connection. Otherwise, open a new connection and read from it. If reading the stream fails, we retry with a new connection. FTr<)r_open_connectionr_read_from_streamrrr|rd)r9r retried_oncers r:r,zHfFileSystemStreamFile.reads ==  ! ! #  $==(D,A,A,I,,T-B-BFKCH$  $==,MM'')%%'#  $sA6;A66AB=<B=iteratorr4c|dk(ry|dkrRt|j}|jj|D]}|j|t |S|t |jkr)t |jd|}|jd|=|St|j}|jj|D]m}|t |z }|t |kDr|j|1|j|d||jj||dt |St |S)zRead up to `length` bytes from stream buffer and stream. If length < 0, read until EOF. If EOF is reached before length, fewer bytes may be returned. rr<N)rrrrrr)r9rrbufchunkresultneeds r:rz(HfFileSystemStreamFile._read_from_stream#s9 Q; A:D//0C    % % '! " 5! ":  S,,- -4..w78F##GVG,M++, !!# ECH$Dc%j  5! 5$<(##**5<8Sz Szr<cL|jj|jSr`rr8s r:rNzHfFileSystemStreamFile.urlErr<cnt|dsy|jjt|Sr)rrrdrarrs r:rzHfFileSystemStreamFile.__del__Hs/t_-   w  r<ct|j|j|j|j|j j ffSr`)reopenrr3r blocksizecacherr8s r:rsz!HfFileSystemStreamFile.__reduce__Os0DIIt~~tzzWWWr<c@|jjd|_|j}|jj j }|jdkDrd|jd|d<|jjtd||tj|_ t|j|jj#|_y#t$r+}|jj dk(r d|_ Yd}~yd}~wwxYw) z%Open a connection to the remote file.NrrrRangerri)rrrrNrrrarr enter_contextr-rrrr+r status_code iter_bytes)r9rNrXrs r:rz'HfFileSystemStreamFile._open_connectionRs !!# $hhj'',,002 88ar?rrAryrbrcr,rrrrNrrsrrrs@r:rrs # !* !*!* !* * !*  !*!*F::S:$3$8 (5/ 3 PU D&S&!X;r<rrHr4cFtj|r|St|Sr`)r#match safe_quoters r:rUrUqs 288B8\ S[H\\r<sct|dS)NrP)safe)r)rs r:rrus  r<r3rc|}t|tr|d}n+t|tr|d}nt|tr|d}t ||)Nz (repository not found)z (revision not found)z (invalid repository id))rrr r*r )r3rmsgs r:rrys\ C#./-. C. /+, C* +./ C c)r<rrrrc,|j||||S)Nr)r+)rr3rrrs r:rrs 774dzj7 QQr<c`||i|}|jD]\}}t||||Sr`)rurv)rdrerfr~rrrs r:rrrrs? d f B+113'kD+&' Ir<)Zrrrl collectionsrcollections.abcrr contextlibrcopyr dataclassesrr r itertoolsr pathlibr r typingrrr urllib.parserrfsspechttpxfsspec.callbacksrrr fsspec.configr fsspec.utilsrrPr _commit_apirrerrorsrrrrr file_downloadr!r"hf_apir#r$r%r&r'r(r)utilsr*r+r,r-r.utils.insecure_hashlibr/r1rErYrrAbstractFileSystemr\rBr^rspecAbstractBufferedFilerrrArUrr|rryrrrhffsrCr<r:rs . ('''' JJ&#C0vvvjj' 66 6 &)A& &0 0%=0 0223 c32l2jB 6,,B J H&v{{77H&VR;V[[==R;j]C]C]##**)d*:*x*R|R3RcRsRPSR~r<