-:j&4$UdZddlZddlZddlZddlZddlZddlmZddlm Z m Z ddl m Z ddl mZddlmZdd lmZiZeeefed <Gd d Zd'd edzddfdZd'd edzdefdZdededzfdZGddeZej=ej>GddZ e de ezdeejBfdZ"ddde#eded edzdefdZ$dddddddddeded edzd!ed"ed#ed edzd$e#edzd%edzdeefd&Z%y)(u Utility helpers to handle progress bars in `huggingface_hub`. Example: 1. Use `huggingface_hub.utils.tqdm` as you would use `tqdm.tqdm` or `tqdm.auto.tqdm`. 2. To disable progress bars, either use `disable_progress_bars()` helper or set the environment variable `HF_HUB_DISABLE_PROGRESS_BARS` to 1. 3. To re-enable progress bars, use `enable_progress_bars()`. 4. To check whether progress bars are disabled, use `are_progress_bars_disabled()`. NOTE: Environment variable `HF_HUB_DISABLE_PROGRESS_BARS` has the priority. Example: ```py >>> from huggingface_hub.utils import are_progress_bars_disabled, disable_progress_bars, enable_progress_bars, tqdm # Disable progress bars globally >>> disable_progress_bars() # Use as normal `tqdm` >>> for _ in tqdm(range(5)): ... pass # Still not showing progress bars, as `disable=False` is overwritten to `True`. >>> for _ in tqdm(range(5), disable=False): ... pass >>> are_progress_bars_disabled() True # Re-enable progress bars globally >>> enable_progress_bars() # Progress bar will be shown ! >>> for _ in tqdm(range(5)): ... pass 100%|███████████████████████████████████████| 5/5 [00:00<00:00, 117817.53it/s] ``` Group-based control: ```python # Disable progress bars for a specific group >>> disable_progress_bars("peft.foo") # Check state of different groups >>> assert not are_progress_bars_disabled("peft")) >>> assert not are_progress_bars_disabled("peft.something") >>> assert are_progress_bars_disabled("peft.foo")) >>> assert are_progress_bars_disabled("peft.foo.bar")) # Enable progress bars for a subgroup >>> enable_progress_bars("peft.foo.bar") # Check if enabling a subgroup affects the parent group >>> assert are_progress_bars_disabled("peft.foo")) >>> assert not are_progress_bars_disabled("peft.foo.bar")) # No progress bar for `name="peft.foo"` >>> for _ in tqdm(range(5), name="peft.foo"): ... pass # Progress bar will be shown for `name="peft.foo.bar"` >>> for _ in tqdm(range(5), name="peft.foo.bar"): ... pass 100%|███████████████████████████████████████| 5/5 [00:00<00:00, 117817.53it/s] ``` N)Iterator)contextmanager nullcontext)Path)ContextManager)tqdm)HF_HUB_DISABLE_PROGRESS_BARSprogress_bar_statesc8eZdZdZddedzddfdZd dZd dZy) disable_progress_barsa& Disable progress bars either globally or for a specified group. This function updates the state of progress bars based on a group name. If no group name is provided, all progress bars are disabled. The operation respects the `HF_HUB_DISABLE_PROGRESS_BARS` environment variable's setting. Works as both a regular call and a context manager: disable_progress_bars() # disables until enable_progress_bars() with disable_progress_bars(): # disables for the block, re-enables on exit ... Args: name (`str`, *optional*): The name of the group for which to disable the progress bars. If None, progress bars are disabled globally. Raises: Warning: If the environment variable precludes changes. NnamereturncF||_tdurtjdd|_yt | |_|t jdt d<yt Dcgc]}|j|ds|}}|D] }t |= dt |<ycc}w)NFzlCannot disable progress bars: environment variable `HF_HUB_DISABLE_PROGRESS_BARS=0` is set and has priority._global.) rr warningswarn_should_reenableare_progress_bars_disabledr clear startswith)selfrkeykeys_to_removes \/var/www/html/tokenscope/api/venv/lib/python3.12/site-packages/huggingface_hub/utils/tqdm.py__init__zdisable_progress_bars.__init__s '5 0 MM~ %*D ! $>t$D D <  % % '-2  *-@_cCNNVZU[[\S]D^c_N_% -', -(-  %`s &BBc|SNrs r __enter__zdisable_progress_bars.__enter__ cH|jrt|jyyr)renable_progress_barsr)rexcs r__exit__zdisable_progress_bars.__exit__s   + !r$r)rr )rN)__name__ __module__ __qualname____doc__strrr"r(r r$rr r ls(*.S4Z.4.(,r$r rrctdurtjdy|tj dtd<ytDcgc]}|j |ds|}}|D] }t|= dt|<ycc}w)a Enable progress bars either globally or for a specified group. This function sets the progress bars to enabled for the specified group or globally if no group is specified. The operation is subject to the `HF_HUB_DISABLE_PROGRESS_BARS` environment setting. Args: name (`str`, *optional*): The name of the group for which to enable the progress bars. If None, progress bars are enabled globally. Raises: Warning: If the environment variable precludes changes. TzkCannot enable progress bars: environment variable `HF_HUB_DISABLE_PROGRESS_BARS=1` is set and has priority.Nrr)r rrr rr)rrrs rr&r&s $t+ y   |!!#)-I&)<[#RVQWWXz@Z#[[! )C#C( )$(D!\s A?!A?ctdury|tjdd S|r8|tvr t| Sdj|j ddd}|r8tjdd S)a Check if progress bars are disabled globally or for a specific group. This function returns whether progress bars are disabled for a given group or globally. It checks the `HF_HUB_DISABLE_PROGRESS_BARS` environment variable first, then the programmatic settings. Args: name (`str`, *optional*): The group name to check; if None, checks the global setting. Returns: `bool`: True if progress bars are disabled, False otherwise. TNrr)r r getjoinsplitrs rrrs~$t+ |&**9d;;;  & &*400 0xx 3,-  #&&y$7 77r$ log_levelc^|tjk(rytjddk(ryy)z Determine if tqdm progress bars should be disabled based on logging level and environment settings. see https://github.com/huggingface/huggingface_hub/pull/2000 and https://github.com/huggingface/huggingface_hub/pull/2698. T TQDM_POSITIONz-1FN)loggingNOTSETosgetenv)r5s ris_tqdm_disabledr<s* GNN" yy!T) r$c6eZdZdZfdZdeddffd ZxZS)rz Class to override `disable` argument in case progress bars are globally disabled. Taken from https://github.com/tqdm/tqdm/issues/619#issuecomment-619639324. ch|jdd}t|rd|d<t| |i|y)NrTdisable)poprsuperr)rargskwargsr __class__s rrz tqdm.__init__s7zz&$' %d + $F9  $)&)r$attrrNcP t||y#t$r |dk7rYywxYw)zBFix for https://github.com/huggingface/huggingface_hub/issues/1603_lockN)rA __delattr__AttributeError)rrErDs rrHztqdm.__delattr__s3  G  % w s %%)r)r*r+r,rr-rH __classcell__)rDs@rrrs% * r$rc@eZdZdZdZdZdZd deezdzddfdZ y) silent_tqdmz#Fake tqdm object that does nothing.cyrr )rrBrCs rrzsilent_tqdm.__init__ r$c|Srr r!s rr"zsilent_tqdm.__enter__ r#r$cyrr )rexc_type exc_value tracebacks rr(zsilent_tqdm.__exit__rNr$nNrcyrr )rrTs rupdatezsilent_tqdm.updaterNr$)) r)r*r+r,rr"r(intfloatrVr r$rrLrLs0-   e d* 4 r$rLpathc#vKt|tr t|}|jd5}|j j }t dd|d|j|jd dtdzdtffd }||_|jdddy#1swYyxYww) uQ Open a file as binary and wrap the `read` method to display a progress bar when it's streamed. First implemented in `transformers` in 2019 but removed when switched to git-lfs. Used in `huggingface_hub` to show progress bar when uploading an LFS file to the Hub. See github.com/huggingface/transformers/pull/2078#discussion_r354739608 for implementation details. Note: currently implementation handles only files stored on disk as it is the most common use case. Could be extended to stream any `BinaryIO` object but we might have to debug some corner cases. Example: ```py >>> with tqdm_stream_file("config.json") as f: >>> httpx.put(url, data=f) config.json: 100%|█████████████████████████| 8.19k/8.19k [00:02<00:00, 3.72kB/s] ``` rbBTr)unit unit_scaletotalinitialdescsizeNrcL|}jt||Sr)rVlen)rcdataf_readpbars r _inner_readz%tqdm_stream_file.._inner_read7s!$rsBH  $2!&4(*T#t)_)/,/,d)sTz)T)@8S4Z848:  t 84 oioo    *4#:*(23D3D*E**ZUY5h5C5sTz5go54(,!  :      *X%d{Dr$